MULTI-SECTOR COORDINATED SUPPORT - TRiPtripregina.ca/wp-content/uploads/2017/10/FINAL... · The Regina Intersectoral Partnership: Final Evaluation Report 3 7.1.9 Case Closure p.42

MULTI-SECTOR COORDINATED SUPPORT An In-depth Analysis of The Regina Intersectoral Partnership’s Integrated

Approach to Reducing Vulnerability Among Children and Youth

FINAL EVALUATION REPORT

Prepared for:

Prepared by: Dr. Chad Nilson

[email protected] 306-953-8384

October 2017

The Regina Intersectoral Partnership: Final Evaluation Report 1

The Living Skies Centre for Social Inquiry is a research, evaluation, and advisory firm with expertise in multi-sector collaborative initiatives designed to improve community safety and well-being. For further information, please contact: Dr. Chad Nilson, Principal Living Skies Centre for Social Inquiry 306-953-8384 [email protected] Acknowledgements: Special thanks to Keri Okanik (LSCSI); Chantel Listrom (TRiP); and all staff and partners of TRiP for assistance in data collection, storage, access, and ongoing support to the evaluation process. This final evaluation report has been prepared by the Living Skies Centre for Social Inquiry at the request of The Regina Intersectional Partnership. For further information on TRiP, please contact: Lance Dudar, Coordinator The Regina Intersectoral Partnership (TRiP) 1600 – 4th Ave Regina, SK 306-523-3024 [email protected] or Wendy Stone Crime Prevention Strategist Regina Police Service Regina, SK 306-777-6646 [email protected] To reference this report, please use the following cite: Nilson, C. (2017). Multi-Sector Coordinated Support: An In-depth Analysis of The Regina Intersectoral

Partnership’s Integrated Approach to Reducing Vulnerability Among Children and Youth (Final Evaluation Report). Prince Albert, SK: Living Skies Centre for Social Inquiry.


TABLE OF CONTENTS 1.0 INTRODUCTION p.4 2.0 BACKGROUND p.6 2.1 Early Work p.6

2.2 Current Focus p.6 2.3 TRiP’s Multi-Sector Team p.7 2.4 Governance p.9 2.5 Function p.9 2.5.1 The TRiP Process p.9 2.5.2 In Support of the Process p.12 2.6 Program Theory p.12

3.0 LITERATURE REVIEW p.14 3.1 Understanding Evaluation p.14 3.2 Lessons Learned from Past Evaluations p.14 3.3 Best Practices in Evaluation for Multi-Sector Collaborative Initiatives p.16 3.3.1 Selecting Variables p.16 3.3.2 Using Evaluation Tools p.17 3.3.3 Client and Stakeholder Focus on Improvement and Capacity-Building p.17 3.3.4 Disseminating Evaluation Findings p.17 4.0 EVALUATION PLANNING p.19 4.1 Assessment of Outcome Linkages p.19 4.2 Evaluation Questions p.19 4.3 Evaluation Activities p.20 5.0 METHODOLOGY p.22 5.1 Program Reporting Data p.22 5.2 Primary Evaluation Data p.25 5.3 Evaluation Matrix p.26 6.0 INTERIM RESULTS p.22 6.1 Preliminary Results p.29 6.2 Secondary Results p.30 7.0 FINAL RESULTS p.32 7.1 Internal Reporting Process p.32 7.1.1 Community Networking and Outreach p.32 7.1.2 Referrals p.34 7.1.3 File Status p.34 7.1.4 Achieved Target Group p.34 7.1.5 Sector Involvement p.37 7.1.6 School Engagement p.39 7.1.7 Community Engagement p.40 7.1.8 Case Duration p.41


7.1.9 Case Closure p.42 7.1.10 Client Outcomes p.42 7.2 Primary Data Results p.45 7.2.1 Client Exit Survey Feedback p.45 7.2.2 Youth Exit Survey Feedback p.46 7.2.3 Caregiver Exit Survey Feedback p.47 7.2.4 Steering Committee Survey Feedback p.52 7.2.5 Coordinated Case Conference Participant Feedback p.56 7.2.6 TRiP Staff Feedback p.61 7.2.7 Evaluator Observations p.72 8.0 SCHOOL ENGAGEMENT OFFICER: SPECIAL ANALYSIS p.74

8.1 About the Position p.74 8.2 Case Involvement p.75 8.3 Client Outcomes p.76 8.4 Feedback p.79 8.5 School Engagement Officer Special Analysis Results Summary p.82

9.0 SUMMARY OF RESULTS p.84 9.1 Internal Reporting Process Results Summary p.84 9.2 Primary Data Collection Results Summary p.84 10.0 OVERALL FINDINGS p.86 10.1 Target Group p.86 10.2 Process p.86 10.3 Satisfaction p.88 10.4 Efficiency p.88 10.5 Effectiveness p.88 10.6 Outcomes p.88 11.0 LIMITATIONS p.89 12.0 CONCLUSION p.90 13.0 RECOMMENDATIONS p.91 References Appendix A Appendix B


MULTI-SECTOR COORDINATED SUPPORT: An In-depth Analysis of The Regina Intersectoral Partnership’s Integrated


EXECUTIVE SUMMARY

Project Background

By focusing on multi-sector coordinated support, reduction of barriers to pro-social activities, and school engagement, both the 11UI and twelve&up components of TRiP aim to generate long-term risk reduction through improving protective factors. The ultimate goal of this process is to reduce vulnerability among at-risk children, youth, and their families in Regina.

Evaluation Activities

• Implementation of new reporting system that allows for improved case management and analytical capacity.

• Observations of integrated case support for vulnerable children and their families.

• Interviews with human service providers and key stakeholders.

• Exit surveys to children, youth, and caregivers.

• Analysis of behaviour-change data from multiple sector inputs.

Method-ology

• Industry-first Risk-Based Needs Assessment Tool that collates data from multiple human service sectors to calculate an aggregate perspective on risk reduction.

• Development of quantitative measurement of proxy for aggregate reduction in vulnerability.

• Full suite of ongoing performance monitoring and client tracking tools.

• Interviews/surveys with TRiP staff, human service providers, managers, clients, and caregivers.

Key Findings • Effective at engaging hard-to-reach clients who are most-vulnerable to crime, violence, school absenteeism, disruptive behaviour, and substance use.

• The leading risk factors TRiP has detected and coordinated integrated support for include: anti-social behaviour, criminal involvement, mental health, physical violence, and parenting concerns.

• There has been a direct reduction of personal, situational, financial, and systemic barriers to service access, community engagement, and support.

• Of approximately 217 clients, 58.6% were engaged in services; 76.7% overcame service barriers; 48.8% engaged in pro-social community activities; 54.1% reduced parenting concerns; and 62.6% showed an improvement in behaviour*.

• Among 53 clients with school attendance problems (and available data), 64.2% achieved either “improved” or “good” attendance. In contrast, only 5.7% clients had “poor” or “sporadic” attendance following TRiP support.

• Among the 148 clients with available data, 82% show a “moderate” to “strong” reduction in aggregate vulnerability.

• Client feedback indicates children/youth were able to engage in supports/activities they would not have the chance to otherwise.

• Caregiver feedback indicates an increase in child/youth confidence and self-control, as well as a reduction in aggression, violence, and anti-social behaviour.

Recommend-ations

build capacity for post-TRiP C4 support ● develop mechanism for systemic issue reporting ● condense ICT meetings ● secure full-time sector representatives ● expand sectors represented at TRiP ● develop capacity for mentors ● depart from 11UI and twelve&up branding and distinction ● remove age limits for clients ● implement the Hub Model within TRiP partnership framework ● expand TRiP to be whole-of-system collaboration catalyst

* For the purposes of this executive summary, 217 is the calculated average N of clients per data sample. It is not the N for each sample

reported herein.


MULTI-SECTOR COORDINATED SUPPORT An In-depth Analysis of The Regina Intersectoral Partnership’s Integrated


FINAL EVALUATION REPORT

Prepared by: Dr. Chad Nilson

Living Skies Centre for Social Inquiry

1.0 INTRODUCTION The Regina Intersectoral Partnership (TRiP) is a three-component initiative designed to improve community safety and well-being in Regina, Saskatchewan. These components include the 11 and Under Initiative (11UI), the Twelve and Up Program (twelve&up), and the Hub Model. By focusing on multi-sector coordinated support, reduction of barriers to pro-social activities, and school engagement, both 11UI and twelve&up aim to generate risk reduction through improving protective factors. The ultimate goal of this process is to reduce vulnerability among at-risk children, youth, and their families. The Hub Model of collaborative risk-driven intervention is designed to identify situations of acutely-elevated risk, and rapidly mobilize interventions of support to prevent crises and harm. While 11UI and twelve&up are currently being implemented, TRiP’s application of the Hub Model is currently still in development. In 2015, TRiP approached the Living Skies Centre for Social Inquiry to conduct an evaluation of the two active components of the model: 11UI and twelve&up. During the summer of 2015, consultations between TRiP and the evaluator highlighted the priorities for the evaluation. These included measurements of program activities, outputs, and outcomes; as well as benefits of this process to clients, human service providers, and their agencies. In the autumn of 2015, TRiP staff worked with the evaluator to further conceptualize the model, identify pertinent evaluation questions, and reveal opportunities for data collection. During this period, the evaluator worked with TRiP staff to develop and implement a new reporting process that not only captures important evaluation and performance monitoring data, but provides TRiP with a more efficient and effective process of identifying client needs, mapping community assets, integrating services, and coordinating supports for the client and his/her family. Between October 1, 2015 and May 30, 2017, qualitative and quantitative data were gathered by TRiP staff using the new TRiP reporting process. Additional data were collected by the evaluator through interviews, surveys, and observation administered by the evaluator. The collection of these data contributes to this evaluation’s examination of process, satisfaction, efficiency, effectiveness, benefits, collaboration, impact, outcomes, challenges, potential improvements, and next steps for TRiP.


Throughout the evaluation period, TRiP staff received an Initial Evaluation Update in May of 20161, as well as a Secondary Evaluation Update in October of 20162. The purpose of these deliverables was to provide an interim report on the progress of data collection, as well as to offer some early analytical findings, troubleshooting within the reporting process, and preliminary recommendations on the design, structure, and process of TRiP. This evaluation report begins with an overview of the TRiP model—with particular attention being placed upon the 11UI and twelve&up initiatives. Following this, is a literature review, which informs this evaluation on past measurements of multi-sector coordinated support programs. Next, is an introduction to the evaluation questions and activities that have guided this project. The fifth section in this report presents the methods of data collection and analysis used to generate answers to these evaluation questions. The 6th and 7th sections of this evaluation report present the interim and final evaluation results, respectively. Following this, there is a special analysis of the School Engagement Officer pilot position within TRiP. The 9th section of this report discusses overall findings, which is then followed with sections describing limitations, concluding observations, and recommendations to improve, expand, and sustain TRiP’s multi-sector collaborative support initiative.

1 Nilson. C. (2016). 11 & Under and Twelve & Up: Initial Evaluation Update. Prince Albert, SK: Living Skies Centre for Social Inquiry. 2 Nilson. C. (2016). 11 & Under and Twelve & Up: Secondary Evaluation Update. Prince Albert, SK: Living Skies Centre for Social Inquiry.


2.0 BACKGROUND At its core, TRiP is a shared commitment by multiple human service agencies to improve client outcomes through intersectoral collaboration, barrier reduction, risk mitigation, and coordinated service provision. By implementing the 11UI and twelve&up initiatives, TRiP aims to make appropriate connections and referrals for children and youth, which will optimize their health, safety and development. By improving communication and collaboration among human service providers, TRiP works to minimize behaviours or conditions that place children in a position of vulnerability (TRiP, 2016). The partners involved in TRiP include Regina Public Schools, Regina Catholic Schools, the Regina Police Service, the Saskatchewan Ministry of Social Services, Saskatchewan Ministry of Justice, and Regina Qu’Appelle Health Region. Each partner provides a staff person, cash and in-kind contributions to the initiative. All partners involved in TRiP are represented on the Steering Committee.

2.1 EARLY WORK The early work of TRiP started with the launch of 11UI in 2010. By design, 11UI focused on providing mentoring and support to children under 11 years of age who were exposed to risk factors that may increase their tendency to engage in unhealthy behaviour. The need for this initiative stemmed from a variety of systemic challenges, including a lack of service coordination, disjointed case management, fragmented information sharing, and an absence of multi-sector collaboration in Regina (TRiP, 2014). Additional rationales for this initiative include a lack of parent involvement, client distrust for service providers, and both personal and financial barriers to service access. The original implementation of 11UI aimed to build the strength and resilience of families by increasing awareness and access to existing social and community support services. Its focus on early collaborative intervention and support was influenced by overwhelming research and evidence-based practice, community sector challenges, and provincial policy alignment concerning community safety and well-being in Saskatchewan (11 & Under Initiative, 2014). To measure the impact of their early undertakings, TRiP participated in an impact evaluation (Wright, 2015) that focused on five outcome areas. These included crime reduction and prevention, student achievement, pro-social engagement, increased coordination of service response, and enhanced family services. Findings of that evaluation revealed that youth involved in the program had a significant decrease in contact with police, along with an observed reduction in the frequency of school absences per month. That evaluation also highlighted a need for continued work in the outcome area of pro-social engagement. At the time of that evaluation, measurements were not available to assess changes in the coordination of service response or enhanced family services.

2.2 CURRENT FOCUS In 2014, TRiP began planning for implementation of their twelve&up initiative. This component of the model was designed to provide similar services as 11UI. However, moving forward in early 2015, both components of the model were enhanced with the addition of a School Engagement Officer. By the fall of 2015, TRiP had reconfigured much of how multi-sector collaborative supports would be delivered to clients of both the 11UI and twelve&up initiatives.


As an upstream, pragmatic approach to improving the safety and well-being of Regina families, the new designs of 11UI and twelve&up were shaped by four guiding principles: pragmatic intersectoral approach; targeted early intervention; family-centred integrated supports; and voluntary participation. In pursuit of these principles, the focus of these two initiatives is on early intervention, intersectoral collaboration, and both family and parental engagement. According to TRiP, early intervention involves providing a wide range of supportive services to increase individual stability, enhance educational achievement and support the resilience of families. Intersectoral collaboration involves an efficient and sensitive coordination of resources that contribute to an informed and assertive coordinated case management approach to supporting individuals and families with composite needs. Finally, family and parental engagement involves building a role for parents in the case planning and resiliency-building activities that support their family (TRiP, 2014). Under the current focus, 11UI and twelve&up support children and youth who are exhibiting behaviors or experiencing conditions that increase their overall vulnerability for harm. While 11UI supports children under 12 years of age, twelve&up supports youth between the ages of 12 and 18. The main objectives of these initiatives are to improve communication and collaboration among service providers, create a process of seamless referrals, foster early risk detection, and engage children in appropriate services, community activities, and pro-social opportunities. The intended outcomes of 11UI include pro-social engagement, school retention, service coordination, enhanced family services, and crime prevention. The intended outcomes of twelve&up are identical, with the exception of one additional outcome: reduced recidivism.

2.3 TRIP’S MULTI-SECTOR TEAM The team responsible for implementing the 11UI and twelve&up initiatives consists of a coordinator, initiative strategist, administrative support, referral and intake officer, case manager, school engagement officer, 2 school liaisons, 3 sector representatives, 1 post-secondary intern, and 2 casual support workers. The coordinator, administrative support, casual support worker, and case manager positions are paid positions through TRiP. The initiative strategist and referral and intake officer positions are provided in-kind from the Regina Police Service. The school engagement officer position is seconded from Regina Police Service through a pilot project funded by an external grant. Both Regina Public School Division and Regina Catholic Schools provide a school liaison to TRiP. Similarly, Saskatchewan Ministry of Social Services, Saskatchewan Ministry of Justice, and Regina Qu’Appelle Health Region each provide a sector representative to TRiP. Finally, the post-secondary intern is provided through the University of Regina. The team is centrally-housed in a shared office setting provided by Regina Public School Division. Its service area is the city of Regina. Although TRiP staff are members of the same integrated team, they are governed by the safety protocols, privacy frameworks, disclosure practices, confidentiality standards, and general policies and procedures of their individual sectors and mandates. To provide an understanding of each position’s contribution to the TRiP Model, Table 1 summarizes some of the key responsibilities of each TRiP team member.


Table 1. Responsibilities of TRiP Team Members by Position

POSITION RESPONSIBILITIES

Coordinator - oversee operations of 11UI and twelve&up initiatives - maintain budget and report on spending - key contact to steering committee and project funders - main spokesperson for TRiP - develop and maintain community partnerships - oversee and support evaluation process

Initiative Strategist

- facilitate strategic visioning and formalization of process and practice - lead development of reports, outreach materials, and knowledge products - coordinate communications and key messaging - oversee instrument development, data collection, and reporting process - conduct community outreach and presentations - support the coordinator in reporting

Administrative Support Specialist

- maintain client database - organize and manage internal reporting and data collection process - advise case leads on upcoming expiration dates or case activities - manage and adjust client wait list

Case Manager - organize and coordinate integrated support planning for clients - follow-up with families and human service providers - maintain oversight and continuity of the ongoing report process - lead/assist in development of Custom Coordinated Case Conference Action Plan - ensure proper case closure reporting and client follow-up

Intake and Referral Officer

- receive and direct referrals - make initial contact and lead intake process with new clients - liaise with referral source and family - complete background checks and risk assessment - serve as police representative in TRiP - present new clients at Intersectoral Collaboration Team meetings

School Engagement Officer

- monitor school attendance and engagement among TRiP clients - provide mentoring and support to children and youth - build incentive plans for school attendance and engagement - work with families to reduce barriers to school engagement, attendance, and performance - form positive relationships between police and at-risk children and youth - participate in and support custom coordinated case conferences

School Liaison - represent division and relevant schools at TRiP - consult with school staff and related professionals on role of TRiP - complete School Background Reports and gather information from schools - support clients and their families in reducing barriers to pro-social activities - arrange custom coordinated case conferences with external community partners

Sector Representative

- serve as point of contact and lead TRiP representative for home agency - pre-screen referrals from home agency to TRiP - participate in weekly meetings to review and discuss referrals - share information and knowledge on client needs, barriers, and service engagement - arrange and lead custom coordinated case conferences with families, schools, and other human service providers - make appropriate referrals for support


Support Worker

- conduct outreach to support clients and their families in participating in pro-social opportunities in the community - encourage school attendance and engagement - encourage appropriate levels of caregiver supervision - serve as a mentor and positive role model to clients and their families

Post-Secondary Intern

- support TRiP staff in client engagement - assist TRiP’s administrative support with internal reporting and data collection process - observe and reflect on integrated services and supports facilitated by TRiP

2.4 GOVERNANCE

TRiP is governed by a steering committee made up of executive leaders from the participating agencies. These include Regina Police Service, Saskatchewan Ministry of Justice, Regina Public School Board, Regina Catholic Schools, Regina Qu’Appelle Health Region, and Saskatchewan Ministry of Social Services. The role of TRiP’s steering committee is to provide strategic vision and organizational guidance to TRiP’s Intersectoral Collaboration Team. On a monthly basis, the steering committee meets to review progress, monitor performance, receive updates, oversee spending, and assist in both trouble-shooting and long-term planning. The steering committee is led by a chair person. The position of chair is determined by an expression of interest. Terms for chairing the steering committee last one year, with the opportunity for extension.

2.5 FUNCTION With respect to function, team members engage in a variety of activities, including: promote TRiP; solicit referrals; identify clients needs and barriers; collaborate with partner agencies; and both engage and support clients and their families. Much of TRiP’s function can be explained through two types of activities. Those that are ‘part of the process’, and those that are ‘supportive of the process’. 2.5.1 The TRiP Process Concerning the former, the TRiP process begins with a referral from caregivers, schools, police, health, or other human service providers. Referrals are received for children and youth showing signs of increased vulnerability and risk. In particular, there are five elements considered in making a referral to TRiP: multi-faceted behavioral changes; composite risk factors; little progress with previous service arrangements; evidence of personal, situational, or institutional barriers to services and supports; and a demonstrable reason that TRiP is the best option to support the client. While the Caregiver Referral Form requests basic information on the client’s needs and current supports, the Agency Referral Form requests information on relevant risk factors, past efforts made to support the client, barriers to service, and previous service involvement. The intake process begins when the Intake and Referral Officer makes initial contact with the client’s caregivers to discuss the referral to TRiP, the core program components, opportunities for support, and their overall interest to enroll their child in the 11UI or twelve&up program. If the caregiver wishes to have their child supported by TRiP, staff arrange for the caregiver’s written consent to share information about their child. Moving forward, the client’s needs, risks, strengths, behaviours, challenges, barriers, home life, community life, school engagement, service history, vulnerabilities and interests are explored through interviews with caregivers, the client, and schools. Critical tools in this process are the Caregiver Intake Guide, Child and Youth Intake Guide, and School Background Report.


The final stage of the intake process is the TRiP Needs-Based Risk Assessment Tool. The tool is not intended to replace professional human service provider discretion, but rather is used as a guide by TRiP staff to determine the overall level of risk to new candidates, relevant to other candidates. This tool uses a risk review and scoring method to identify the need complexity and severity of TRiP candidates. Using aggregate data from the Agency Referral Form, Caregiver Intake Guide, Child and Youth Intake Guide, and School Background Report, TRiP staff complete the tool and tabulate a Total Needs Score from the weighted values given to each risk variable. The needs score is then interpreted in one of three ways: 0 to 5 = ‘could benefit from support’; 6 to 19 = ‘would benefit from support’; and 20 or higher = ‘needs significant support’. It is important to note that not all candidates referred to 11UI or twelve&up are accepted. On occasion, candidates are declined for a number of possible reasons: ‘scored low on Risk-Based Needs Assessment’; ‘referring agent did not articulate that they have explored other options’; ‘services are currently being accessed’; and ‘multiple partners appear to already be actively involved in supporting the child/family’. Once a client is accepted into either the 11UI or twelve&up program, their intake package is summarized and submitted for review at a weekly meeting of TRiP’s Intersectoral Collaboration Team (ICT). During this meeting, TRiP staff identify appropriate resources and supports in the community that may contribute to integrated case management. Also during this meeting, an appropriate case lead is identified among ICT members. To formalize TRiP’s commitment to collectively supporting the client, an ICT Action Plan is completed. This form captures client risk factors, barriers, services to be engaged, relevant support assets, and suggested actions. Following the ICT’s review of a client’s intake package, the case lead will contact the caregiver and mobilize appropriate partners to support the family. In partnership with the caregiver, a Coordinated Custom Case Conference (C4) is scheduled to strategize mitigation of client needs and reduction of barriers. During this process, all agencies, including the caregivers, work towards a consensus on what the best course of action would be to reduce vulnerability of the client. Upon reaching consensus, all C4 participants commit to their roles and responsibilities in supporting the client and his/her family. The initial C4 meeting concludes with an agreed-upon schedule and list of priority actions. During ongoing C4 meetings, whenever they are scheduled, the case lead completes a C4 Report that tracks agency participation, client progress, barrier reduction, and next steps for each relevant support asset. Other than age (i.e., 18 years), there is no fixed threshold for when TRiP discontinues its integrated support of a client. Essentially, the C4 team continues to work with a client and their family until vulnerability subsides, and/or the group (including caregivers) feels comfortable closing the child’s file. Besides ‘reduced vulnerability’, other reasons for case closure include ‘refusal of services’, ‘relocated’, ‘not engaged’, ‘family not supporting school attendance’, and ‘other’. When a TRiP file is closed, the case lead completes a Case Closure Report, which reports on mitigated risk, service provision, school engagement, barrier reduction, family life, behavior change, condition modification, and vulnerability reduction. Additional end-of-service instruments administered include the Caregiver Exit Survey, Child Exit Survey, Youth Exit Survey, and Post-TRiP School Background Report. Collectively, these instruments measure client/caregiver satisfaction, progress, program impact, client outcomes, strengths, weaknesses, and opportunities for improvement.


To support understanding of this process, TRiP has created a Process Flow. As shown in Figure 1, the main stages of the 11UI and twelve&up process include referral, intake, initial contact with family, intersectoral collaboration team, coordinated custom case conference, and conclusion.

Figure 1. 11UI and twelve&up Process Flow

(source: TRiP, 2016)

To illustrate the client experience in the above-described, the following vignette provides a summary of Victoria (fictitious character). This experience is quite typical of the clients who are supported by TRiP.

Victoria (not her real name) was a twelve-year-old female who was referred to the twelve&up program by her school Principal in December 2015. Concerns noted at the time of the referral included gang association and some indication that she wanted to join a gang, negative peers, exposure to drug use, confrontations with peers and adults, oppositional behaviour, and she lacked the ability to develop healthy relationships. She took pride in intimidating others and had expressed a desire to harm peers as well as herself. While still eleven years old Victoria was arrested after being involved in stealing and crashing a car. Victoria’s mom was thirteen when she had Victoria. In the home environment Victoria was exposed to drug and alcohol use as well as domestic violence. Victoria’s biological father left and there have been other father figures in and out of Victoria’s life. A person Victoria was close to committed suicide when Victoria was eight years old which had a significant impact on her.

After receiving Victoria’s referral, the TRiP Intake and Referral Officer met with mom and received signed consent to work with Victoria. Victoria was presented at the iCT (Intersectoral Collaboration Team) meeting in March 2016 and was accepted into the program. It was identified at ICT that here had been some previous Ministry of Social Services involvement. It was


also identified that Victoria had been involved with the School counsellor. At iCT it was decided that there would be involvement from the TRiP Case Manager and the School Engagement Officer (Police). The School Counselor (Public), Principal, Vice-Principal, her teacher and her mom would be invited to the Coordinated Custom Case Conference (C4) meetings.

Victoria was offered the opportunity to attend various pro-social outings with the School Engagement Officer, including but not limited to: working out together at a gym, Regina Police Service Showcase, boxing classes, museum visits, and baking classes. The School Engagement Officer also attended events that Victoria participated in such as basketball games, track and field, and her grade 8 graduation. Victoria has been provided the opportunity to attend guitar lessons, and basketball skill development camps. Victoria has also been connected to private counseling services. Victoria’s Case Manager and School Engagement Officer maintained regular contact with her mother, and school personnel over the following number of months.

Victoria is surrounded with a support system that encourages her to continue to make better life choices. She is involved in pro-social activities that help build her confidence and social skills. Victoria’s attitude toward education has changed and she is doing much better at school. She received two awards at her grade eight graduation, one for literacy and one for junior leadership. There has been no police contact since her involvement with TRiP started. Victoria’s situation has stabilized. TRiP will continue to be involved over the summer and will conclude her file once a successful transition to high school is made in the fall of 2017.

2.5.2 In Support of the Process In support of the process, TRiP staff carry out a variety of additional activities. One major activity is promoting the model throughout the community. This is accomplished through partner agency outreach, presentations, and information sharing from TRiP staff to their home agencies. Another major activity is the ongoing reporting and data collection process. This activity helps to not only better manage clients needs and progress, but enables performance monitoring and outcome measurement. A third activity in support of the process is threefold: ongoing partnership development, community involvement and human service networking. This activity helps to identify additional community assets that may be available to support TRiP clients.

2.6 PROGRAM THEORY In the practice of evaluation, mapping program theory in a logic model is a common and useful exercise. Essentially, the different components of a program, the undertakings of its staff, and the objectives it hopes to achieve are drawn out within the context of inputs, activities, outputs and outcomes. As Figure 2 illustrates, implementation of TRiP’s 11UI and twelve&up program require several inputs. These include staff, community partners and office space. The activities explored in this evaluation include community outreach, receipt of referrals, targeted single intake, needs identification, service mobilization, coordinated custom case conferencing, support for children, support for caregivers, and barrier reduction. The outputs generated by these activities—including community awareness, client referrals, new clients, identified needs, service involvement, coordinated action plans, support for clients, support for caregivers, and removal of barriers—trigger a sequence of short-term and intermediate outcomes, resulting in reduced vulnerability among children and families. The ultimate outcome of this reaction is improved community safety and well-being in Regina.


Figure 2. 11UI and twelve&up Logic Model


3.0 LITERATURE REVIEW This literature review was conducted in support of building a sound methodology for the evaluation of TRiP’s 11UI and twelve&up initiatives. The information gleaned from the literature review process has been used to influence the evaluation design and implementation; provide direction and insight to assess strengths; and develop evidence-based, action-oriented recommendations for capacity-building in order to improve TRiP’s multi-sector coordinated support initiatives. 3.1 UNDERSTANDING EVALUATION Evaluation refers to the careful examination of the processes and outcomes of a service/program (Fitch & Grogan-Kaylor, 2012; Taylor-Powell, 2009). Evaluating multi-sector collaborative interventions and ongoing supports requires a complex evaluation methodology capable of capturing information about a variety of evaluation variables from multiple data sources. Comprehensive evaluation involves measuring program activities, outputs, and outcomes. As such, evaluation must incorporate both process and outcome evaluation techniques. According to the Addiction and Mental Health Collaborative Project Steering Committee (2015), process evaluation monitors and documents specific aspects of implementation to determine the relationships between elements of the initiative and outcomes produced, whereas outcome evaluation assesses whether the activity/initiative has had an impact on targeted outcomes. Outcomes are the changes in the client’s life that have occurred as a result of their interaction with the activity/initiative. According to Fitch & Grogan-Kaylor (2012), evaluative activities should seek data to answer two important questions:

• Does the initiative/program follow a coherent program logic? (process evaluation)

• Do the individual programs/interventions within the organization appear to demonstrate clinical effectiveness? (outcome evaluation)

Using a logic model as the basis for evaluation allows evaluators to visually depict the inputs/resources, activities, and intended outcomes of the program/intervention. Renger, et al. (2015), Roberts & Yeager (2004), Rush (2003) and Orwin (2000), suggest that a logic model supports the evaluation process by providing a conceptual blueprint for determining what to assess and how to go about it. It provides the evaluation with direction and keeps evaluation methods and tools consistent with the goals and objectives of the program/intervention. TRiP’s logic model (see Figure 2) provides a clear structure of activities, outputs, and anticipated outcomes to form the basis of the evaluation of the 11UI and twelve&up initiatives. 3.2 LESSONS LEARNED FROM PAST EVALUATIONS While 11UI and twelve&up are focused primarily on children and families, much of the existing evaluation literatures on intervention and ongoing coordinated support focus on those interventions designed and delivered to adult populations. However, evaluation findings from adult programs can often teach us valuable information about best practices in collaborative initiatives and offer insights into their potential benefits. For example, in their review of the service agreements between Correctional Service Canada and the New Brunswick Department of Public Safety (Correctional Service of Canada, 2008), evaluators determined that multi-sector collaborative intervention and support initiatives have several benefits, including greater interagency awareness and cooperation, and increased access to and provision of programs to clients.


In an evaluation of multi-sector collaborative interventions provided to adult offenders, Correctional Service Canada (2012) discovered that there are several key ingredients necessary for multi-sector collaborative initiatives to be successful. These include: strong communication between service providers involved in the initiative; a focus on partnerships; and providing access to a wide range of community services and supports that are specifically aligned to individual client risks and needs. In preparation for the evaluation of TRiP’s 11UI and twelve&up initiatives, the evaluator reviewed evaluation literature of other multi-sector collaborative program models in Canada, including: The Hub Model of collaborative risk-driven intervention (e.g., Babayan, Laundry-Thompson, & Stevens, 2015; Gray, 2015; Litchmore, 2015); police and mental health crisis teams (e.g., Belleville Police Service, 2007; Chandrasekera & Pajooman, 2011); restorative justice programs for both youth and adults (e.g., Bonta et al., 2010; Wilson et al., 2009; Latimer et al., 2001); court diversion programs and problem-solving courts for both youth and adults (e.g., Werb et al., 2007; Hornick et al., 2005; Fischer & Jeune, 1987); The Integrated Police and Parole Initiative (e.g., Gossner et al., 2016; Razansoff et al., 2013); adult offender reintegration programs (e.g., Cherner et al., 2014; Bellmore, 2013; British Columbia Ministry of Public Safety, 2011); community programs delivered in incarceration facilities (e.g., Andrews et al., 1990; British Columbia Corrections Research Unit, 2009; Lafortune, 2015); police youth outreach programs (e.g., Augmeri et al., 2007; Lipman et al., 2008; Cooper, 2014); Aboriginal partnerships (e.g., Government of Canada, 2015; Hubberstay et al., 2014; Public Safety Canada, 2014); community safety teams (e.g., City of Calgary, 2010); police prevention initiatives (e.g., Giwa, 2008; Dumaine, 2005; Walker & Walker, 1992); the Drug Abuse Resistance Education Program (e.g., Royal Canadian Mounted Police, 2007; Ringwalt et al., 1991); harm reduction programs (e.g., van der Meulan et al., 2016; Cooper et al., 2005; Kerr et al., 2005); and, multi-sector training and education initiatives (e.g., Krameddine et al., 2013). Reviewing these past evaluations can provide evaluators with insights into which process and outcome variables should be measured in order to adequately assess the successes and challenges of the implementation of a multi-sector collaborative intervention and support process. Table 2 highlights the most common variables measures in past evaluations reviewed for this report.

Table 2. Variables Measured in Past Evaluations of Multi-Sector Collaborative Initiatives Process Evaluation Variables Outcome Evaluation Variables

Agencies participating Activities involved Resources required Number of clients completing/not completing Parent engagement Program fidelity Client demographics Referral source Referrals to outside agencies Perception of program from larger community Opportunities for program growth/improvement Barriers to service access

Level of client engagement (e.g., attendance) Change in client behaviour Change in client risk/reduced vulnerability Reduced rates of recidivism (criminal justice only) Change in community engagement Change in school attendance Change in school achievement (e.g., school engagement and behaviour) Perception of program from service providers Ongoing program implementation challenges Client satisfaction Overcoming family concerns

Engaging in an assessment of past evaluations of multi-sector collaborative initiatives can also be useful in determining the strengths of such programs as well as common challenges they experience. This information can be used to determine how TRiP compares to other similarly-structured interventions. It


provides insights into evaluation techniques to assess common challenges, as well as ensure ongoing program improvement and capacity-building for the 11UI and twelve&up initiatives. Some of the common strengths of multi-sector collaborative initiatives discovered through this literature review include: more rapid access to services and improved responsivity of those services to client needs (Gray, 2016; Clement, 2016; Cherner et al., 2014; Rezansoff et al., 2013); improved information sharing amongst participating organizations and greater interagency awareness (Gossner et al., 2016; Belmore, 2013; Lipman et al., 2008); enhanced community/school engagement (Lafortune, 2015; Cooper, 2014); and, reduced risk/reduced vulnerability of clients and families (Gray, 2016; Kirst et al., 2015; Augimeri et al., 2007). While past evaluations of multi-sector collaborative intervention and support initiatives demonstrate that there is much value in delivering these types of programs to clients, they are not without their challenges. Some of the common challenges highlighted in the evaluation literature of these programs include: varying interpretations of privacy and information-sharing (Nilson, 2015; Fischer & Jeune, 1987); negative perception of initiative by larger community (Clairmont & Waters, 2015; Rugge et al., 2005); and, inconsistency in program delivery/fidelity (Bhayani & Thompson, 2016; Slinger & Roesch, 2010). Learning about the challenges experienced by similar programs and how they were overcome can assist this evaluation to develop an evidence-based and action-oriented set of recommendations for TRiP to consider in overcoming their own challenges. Through this review of past evaluations, the TRiP evaluation team was able to determine key ingredients and evidence-based practices for conducting a comprehensive evaluation of TRiP’s initiatives. The next section of this literature review highlights the best practices gleaned from this review process.

3.3 BEST PRACTICES IN EVALUATION FOR MULTI-SECTOR COLLABORATIVE INITIATIVES Best practices refer to program procedures that have been thoroughly researched and are accepted as being the most effective to accomplish program goals; that is, they have been demonstrated through comprehensive evaluation to produce optimal outcomes. Based on this review of past evaluations of multi-sector collaborative initiatives, four best practices were determined to enhance the evaluation of TRiP’s 11UI and twelve&up initiatives:

• Selecting appropriate variables to be measured

• Using evaluation tools

• Focus on ongoing program improvement and capacity building

• Appropriate dissemination of findings This section of the literature review will highlight key information and processes to ensure the utilization of best practices throughout the TRiP evaluation process. 3.3.1 Selecting Variables A variable is a key characteristic or attribute of a program/participant that evaluation sets out to measure (Korb, 2012). The importance of selecting appropriate variables to be measured through the evaluation process cannot be overstated. As discussed previously in this review, comprehensive evaluation must consider both process and outcome variables. Comprehensive evaluation also involves the assessment of both quantitative and qualitative information. Quantitative information provides a


broad view of a program and its accomplishments and challenges, while qualitative information helps to describe those accomplishments and challenges and provides for a more in-depth analysis of quantitative findings. Using logic modelling and other evaluation tools can help evaluators define what variables are going to be measured through evaluation and ensure that this information is collected from program stakeholders effectively and efficiently. 3.3.2 Using Evaluation Tools Evaluation instruments and tools provide consistency and increase the ability of evaluators to use evaluation data to improve programs/services and organizational decision-making. Reporting tools must be designed to capture information about client risk, parent engagement, community engagement, school engagement and behaviour, barriers to service access, as well as services provided during the intervention and support process. The objective of instrument development for evaluative activities is to translate information needs into a set of specific questions that clients are willing and able to answer. According to Farnik & Pierzchala (2012), Gee et al. (2010) and Newman et al. (1987), there are several key considerations for the development of evaluation tools: validity; sensitivity to change; reliability; ease of use; and respect for the rights of all clients. As the methodology of this evaluation report will show, TRiP has implemented a rigorous data collection and reporting process with these considerations in mind. 3.3.3 Client & Stakeholder Focus on Improvement and Capacity-Building Program improvement and capacity-building involve using evaluation as a means to further develop and enhance programs and services so that they are better able to achieve desired outcomes. This must be seen as an ongoing process of continuous program adaption and improvement to meet the changing needs of clients and communities. As such, evaluation that focuses on program improvement and capacity building must involve the voices of the clients/participants of the program. Clients should be seen as experts of their own lives and experiences. Their experience can provide evaluators with valuable insights to improve program design and delivery and to ensure that the needs of the clients are being met effectively and efficiently. While information about the aspects of the program that were helpful is important, a key element of any comprehensive evaluation is to seek information from clients and other stakeholders about challenges/barriers of the existing program and their ideas for overcoming these barriers in future implementations of the program. 3.3.4 Disseminating Evaluation Findings Having a dissemination plan for evaluation findings ensures that the right information gets to the right people. In order for evaluation to lead to ongoing program improvement and capacity building, the evaluation methodology must include processes to ensure that all findings from the evaluation are reported in an accessible manner and disseminated to the appropriate stakeholders (e.g., clients and their families; program staff; funders; other community services/organizations; and, general community members). When evaluation information is presented to these groups in a meaningful way, it increases the potential to improve relationships amongst program staff and clients, enhances the capacity of service providers to administer future implementations of the program, improves the reputation of the program within the larger community, and enhances support for the initiative’s continued implementation. A critical consideration when disseminating evaluation findings is to link key findings to specific program recommendations to enhance services and build program capacity.


Overall, the information and insights gained from this literature review have been used to guide and influence the design and implementation of this evaluation. The methodology described herein, is consistent with the leading best practices in evaluating multi-sector collaborative interventions and supportive services.


4.0 EVALUATION PLANNING To conduct an evaluation of this scope and depth, considerable planning went into making sure the final evaluation meets or exceeds the needs of TRiP staff and steering committee. To make sure that this process delivered what TRiP stakeholders require, the evaluator worked with TRiP staff, partners, and stakeholders to identify evaluation priorities, potential data collection opportunities, and a process to access, store and analyze the evaluation data. Three main parts of this planning process included the assessment of outcome linkages, development of evaluation questions, and determination of evaluation activities. 4.1 ASSESSMENT OF OUTCOME LINKAGES One of the first steps in preparing for evaluation is to identify the scope of the evaluation, and in particular, the parameters within which the evaluation questions and responding methodology can exist. During a consultation process with TRiP stakeholders, it was determined that the major focus of inquiry for this evaluation should be on verifying the direct client outcomes that are attributable to TRiP. In particular, stakeholders believed that this evaluation should concentrate on individual-level outcomes (e.g. risk) as opposed to system-level outcomes (e.g. crime occurrence, graduation rates). The confidence in moving towards individual-level outcomes stems from the large body of literature that already shows how generation of certain individual-level outcomes contribute to system-level outcomes in the policing/justice, mental health, education, and social work sectors. For example, reducing certain risk factors, such as negative peers, disorganized homes, aggression, impulsivity, school disengagement, poor parenting, and parent conflict, has been shown to reduce the likelihood of criminal behaviour later in life (Shader, 2004). Similarly, building protective factors against such risks as anti-social role models, social isolation, poor school attachment, exposure to violence, and barriers to services, has been shown to reduce the likelihood of children and youth to develop mental health issues (Commonwealth Department of Health and Aged Care, 2011). In the education sector, system-level outcomes are often negatively affected by parental separation, depression, parent substance abuse, aggression, lack of attachment, abuse, and traumatic life events (Huffman, Mehlinger, & Kerivan, 2000). Finally, many of the risk factors affecting family safety that is monitored by the social work sector include young children in the home, special needs, substance abuse, mental health issues, single parenting, low income, transient caregivers, family disorganization, violence, and poverty (Centers for Disease Control and Prevention, 2017). These outcome linkages, supported by past research in multiple sectors, allow the current evaluation to assume that improvements in individual-level outcomes—such as risk reduction, school engagement, behaviour change, and family life improvement (for example)—contribute to broader system-level outcomes. Therefore, the focus of this evaluation will be on verifying qualitatively and quantitatively, the generation of individual-level outcomes among TRiP clients. 4.2 EVALUATION QUESTIONS The next step in evaluation is to develop a set of questions that reflect the key themes of project stakeholders. In the case of the 11UI and twelve&up initiatives, consultations with TRiP staff and the steering committee were used to identify evaluation priorities. During this consultation process, a number of common themes were identified as important to the evaluation. These include achieved


target group, process, satisfaction, efficiency, effectiveness and outcomes. Using these themes, the evaluator led a follow-up discussion with both consultation groups to further refine the evaluation questions. Table 3 provides an overview of the questions driving this evaluation.

Table 3. Evaluation Questions for 11UI and twelve&up by Theme

THEME EVALUATION QUESTIONS

Target Group What is the achieved target group of the initiative?

Process Was the initiative implemented as planned?

What are the critical ingredients to the initiative’s overall implementation?

How does the initiative facilitate collaboration among human service providers?

What are the benefits of the initiative to human service providers?

What challenges were encountered during implementation of the initiative?

What opportunities are there for improving the initiative?

What is required for future growth and expansion of the initiative?

What factors are important to consider in replication of the initiative?

Satisfaction Are children satisfied with their experience with the initiative?

Are parents satisfied with their experience with the initiative?

Are case conference participants satisfied with their experience with the initiative?

Are community stakeholders satisfied with the initiative?

Efficiency Has the initiative increased or expedited client access to services?

What affect has the initiative had on service provider responsiveness to client needs?

Effectiveness Was the initiative successful in coordinating service delivery?

Was the initiative successful in making children feel supported?

Was the initiative successful in making parents feel supported?

Was the initiative successful in reducing barriers (to support and pro-social activities)?

Outcomes Has the initiative influenced a reduction in risk?

Has the initiative influenced a change in behaviour and attitude?

Has the initiative influenced an increase in access to pro-social activities?

What impact has the initiative had on school achievement?*

What impact has the initiative had on pro-social community engagement?

Has the initiative reduce overall vulnerability of the client? *School Achievement includes attendance, academic performance and school involvement.

4.3 EVALUATION ACTIVITIES During the evaluation process, a number of activities are required in order for the planned objectives of an evaluation to be achieved. Some of these activities are the sole responsibility of the evaluator, whereas others require the efforts of staff, steering committee members, or community stakeholders. Table 4 identifies the different activities of this evaluation, including who participated, what the purpose of the activity was, and the time period in which it occurred.


Table 4. TRiP Evaluation Activities: October 2015 – May 2017

Table 2. Engagers, Anticipated Results and Time Period of Evaluation Actions ACTIVITIES PARTICIPANTS PURPOSE TIME PERIOD

Consultations with TRiP evaluation stakeholders

steering committee, staff

more clearly define research questions, data collection opportunities, and presenting themes of evaluation

July – Sept 2015

Evaluation planning staff Identify data sources, collection methods, and reporting processes

Nov – Dec 2015

Research past research and evaluations of risk reduction initiatives

evaluator only Identify methodological insight and lessons learned in past research

Dec 2015 –May 2017

Development of primary data collection instruments

staff create instruments to gather data during evaluation process

Sept – Oct 2015

Ongoing improvement of reporting process

staff troubleshoot, improve, and streamline ongoing data collection

Sept – Nov 2016

Initial data extraction from reporting process

staff opportunity to assess design and implementation of reporting process

Feb – Apr 2016

Preparation and delivery of Initial Evaluation Update

evaluator only provide interim update on evaluation data collection and reporting process

May 2016

Secondary data extraction from reporting process

staff opportunity to assess design and implementation of reporting process

Sep – Oct 2016

Preparation and delivery of Secondary Evaluation Update

evaluator only provide interim update on evaluation data collection and reporting process

Nov 2016

Final data extraction from reporting process

staff prepare evaluation data for final analysis

Mar – May 2017

Collection of survey and interview data

steering committee, staff, partner agencies

gather data for final analysis Mar – May 2017

Analyze primary and reporting data

evaluator only preparation of results for evaluation report

Mar – Jul 2017

Preparation and delivery of Final Evaluation Report

evaluator only prepare and deliver findings to TRiP stakeholders

May – Oct 2017


5.0 METHODOLOGY Evaluation of the 11UI and twelve&up initiative involves a mixed-methods approach. Throughout the evaluation, program data from internal reporting; interviews with staff; observations by the evaluator; and surveys to stakeholders, clients, and caregivers were used to generate data on a number of different indicators. The two main influences defining this methodology include logic model components (e.g., program outputs, short-term outcomes, intermediate outcomes, long-term outcomes) and evaluation questions (e.g., target group, satisfaction, efficiency, effectiveness). Throughout the evaluation, two types of data were gathered: program reporting data and primary evaluation data. The following sub-sections describe each of these data types and their collection method. 5.1 PROGRAM REPORTING DATA One of the more rigorous efforts to collect data in this evaluation was the internal reporting process. In the fall of 2015, the evaluator facilitated a planning session with TRiP staff to determine the different information and data needs of the 11UI and twelve&up initiatives. Part of this process involved exploring the different reasons for data collection, including operational needs, performance monitoring, and evaluation. During the referral, intake and custom coordinated case conference processes, several information exchanges are required to develop and implement an action plan aimed at helping children and their families. As such, the reporting tools were designed to help satisfy operational needs of the initiative. With respect to performance monitoring, certain data points in the reporting tools were included to help maintain continuity, consistency, and fidelity in initiative delivery. Finally, a number of data points in the reporting tools were included to help support the evaluation process. The result of the planning session with TRiP staff was the development of internal reporting tools to meet the operational, performance monitoring, and evaluation needs of the initiative. Each of these tools coincides with a specific activity or occurrence in the 11UI and twelve&up initiatives. Table 5 introduces the different reporting tools by providing a brief description and identifying the user.


Table 5. Internal Reporting Tools REPORTING TOOL DESCRIPTION USER

Community Networking & Outreach Form

A form used to record promotions of TRiP through presentations, major networking opportunities or unilateral outreach to other agencies, government or the public.

Any TRiP Staff

Caregiver Referral Form

For parents/caregivers to complete if they wish their child to be considered for coordinated custom case conference support in the 11UI or twelve&up initiative.

Caregivers/parents

Agency Referral Form

For agencies wishing to make a referral to TRiP. The form includes some criteria to help agencies determine if referral is appropriate.

Referring agencies

School Background Report

Provides an education perspective on the candidate’s family situation, engagement in school, performance, behaviour, and general needs. To be completed by schools during the intake process.

Schools/School Liaisons

Caregiver Intake Guide

A guide used to help the Referral and Intake Officer gather the necessary information to inform the Intersectoral Collaboration Team of candidate needs, services, and barriers.

Referral & Intake Officer

Child & Youth Intake Guide

A guide used to help the Intake and Referral Officer gather the necessary information to inform the Intersectoral Collaboration Team of candidate needs, services, and barriers.


TRiP Risk-Based Needs Assessment

A tool designed to help identify candidate needs; based upon risk information gathered through other reporting tools. Interpretations of the Needs Assessment Score can help determine next actions. Tool also administered at the end of a client’s involvement in TRiP to measure change in vulnerability.


ICT Action Plan A shared account of risk factors, services needed, barriers, logistics, and agencies to involve in C4

Case Lead

C4 Report A written account of the identified client risk factors, agencies involved in the C4, barriers, services to be engaged through the C4; and outputs, challenges and next steps of the C4.

Case Lead

Community Engagement Form

Used to record community engagement support provided by Support Workers or other TRiP staff.

Support Workers/TRiP Staff

School Engagement Summary

A summary sheet for the School Engagement Officer to record ongoing efforts to engage clients in school—with a focus on activities, outcomes and collaboration.

School Engagement Officer

Case Closure Report

A report used to capture the risk factors addressed, services provided, school engagement, community engagement, family concerns; and changes in condition, behaviour and overall vulnerability.

Case Coordinator

Post-TRiP Student Report

A report used to capture changes in parent engagement, as well a student performance, behaviour, engagement, service access and barriers.

Schools/School Liaisons

For this evaluation process, each of the above-listed reporting tools contributes data to help monitor and measure the different activities, outputs and short-term outcomes of TRiP. In order to both assess the extent to which TRiP staff are properly implementing the data collection process, and organize the data collected to date, the evaluator created and launched the TRiP Reporting Database in September of 2015. This database, built in Excel, is managed by the administrative support specialist. Following completion of any of the above-mentioned tools, TRiP staff forward copies of these instruments to the administrative support specialist, who takes specific data from the tools and inputs them onto the


database. Following this, all tools were then scanned and posted onto a secured network drive for the entire TRiP team to access as needed. The description and format of variables captured by the TRiP Reporting Database are summarized in Table 6.

Table 6. Reporting Process Variables Captured in the TRiP Reporting Database

VARIABLE DESCRIPTION FORMAT

Program Type Identified whether client is involved in 11UI or twelve&up. Nominal drop-down

Birth Date Client date of birth in order of month, day, year. Open text

Sex Sex type of client (male/female). Nominal drop-down

Referral Date Date of referral to TRiP in order of month, day, year. Open text

Referral Number A number assigned to clients referred to TRiP. Open text

Referral Source Source of referral to TRiP by sector (public education, catholic education, social services, mental health, police, justice, parents).

Nominal drop-down

Lead Agency TRiP partner assigned as lead in organizing services and support for the client and their family (public education, catholic education, social services, mental health, police, justice, not determined yet).

Nominal drop-down

TRiP Sector Involvement

Identifies whether or not each TRiP sector partner is involved in the case or not (yes/no for each: public education, catholic education, social services, mental health, police, justice).

Nominal drop-down

School Engagement Officer Involvement

Identifies whether the client’s case file involved TRiP’s School Engagement Officer in any of the planning or support (yes/no).

Nominal drop-down

Conference Agency Involvement

Captures the different agencies involved in the Custom Coordinated Case Conferences organized by TRiP.

Open text

Status Provides a brief narrative on the current status of a client’s file. Open text

Conclusion States the reason why a client’s file was closed (reduced vulnerability, potential to reduce vulnerability, aged out, screened out, refused consent, refused services, not engaging, in custody/care, unable to locate, moved away, other).

Nominal drop-down

Reporting Tools Completed

Monitors what reporting tools have been completed and stored in the client’s file (yes/no for each: Caregiver Referral Form, Agency Referral Form, School Background Report, Caregiver Intake Guide, Child & Youth Intake Guide, Needs Assessment, ICT Action Plan, C4 Report, Community Engagement Form, School Engagement Summary, Case Closure Report, Aggregate Risk Assessment, Child Exit Survey, Caregiver Exit Survey, Youth Exit Survey).

Nominal drop-down

Risk-Based Needs Score

A score calculated from the Risk-Based Needs Assessment to identify client’s overall level of risk.

Interval open text

Risk Factors Identification of risk factors affecting client (yes/no to each: alcohol, drugs, gambling, mental health, physical health, suicide, self-harm, criminal involvement, crime victimization, physical violence – perpetrator, physical violence – victim, emotional violence – perpetrator, emotional violence – victim, sexual violence – perpetrator, sexual violence – victim, elderly abuse, lack of supervision, basic needs, missing school, parenting, housing, poverty, negative peers, anti-social behaviour, unemployment, missing/runaway, threat to public safety, gangs, social environment).

Nominal drop-down


Barriers Reports the barriers to service and support that the client encounters at the time of referral (yes/no to each: personal barrier, financial barrier, situational barrier, institutional barrier)

Nominal drop-down

Service Needs Identifies the services that TRiP’s Intersectoral Collaboration Team has identified for the client as needing (yes/no to each: social services, social assistance, housing, mental health, sexual health, public health, medical health, addictions, harm reduction, counselling, cultural support, spiritual support, parenting support, education support, employment support, home care, life skills, victim support, safe shelter, police, courts, corrections, probation, parole, legal support, fire department, mentorship, recreation, food support, other.

Nominal drop-down

Closure Outcomes

Captures a variety of data from the Case Closure Report including service provisions (yes/no to: connected to services, service barriers overcome); community activities (engaged in activities, barriers to activities overcome); family concerns overcome (yes/no); services provided to family (yes/no); positive change in behaviour (yes/no); reduced vulnerability (yes/no); post-TRiP school attendance (improved/same/worsened); post-TRiP school performance (improved/same/worsened).

Nominal drop-down

Once data are entered onto the TRiP Reporting Database, the nominal drop-down data are periodically analyzed using frequency distributions and basic categorical analyses techniques, whereas the open text entries into the database are explored through both content and thematic analysis. 5.2 PRIMARY EVALUATION DATA In addition to the reporting data described in the previous section, primary evaluation data were gathered through surveys, interviews, and observation. In April 2016, two separate surveys were administered. One was completed by TRiP staff, while the other one was completed by human service participants of Custom Coordinated Case Conferences. Results from this first round of surveys were used to provide an initial evaluation update to TRiP staff and stakeholders in May of 2016. Then in March and April of 2017, C4 participants were administered the same survey, while TRiP staff participated in closing interviews. Also at this time, members of the TRiP steering committee were asked to complete a survey that was exchanged electronically with the evaluator. For both the survey and interview process, the sampling strategy was purposeful and based upon the availability and interest of respondents. Other sources of primary data in this evaluation were the three different exit surveys administered to children in the 11UI program, youth in the twelve&up program, and caregivers of each cohort. Following completion and/or withdraw from TRiP support, each cohort was asked by TRiP staff to complete an exit survey. Upon completion by respondents, surveys were then passed along to the administrative support specialist for scanning and digital uploading. In addition to the interviews and surveys conducted, supplementary primary data was gathered through the evaluator’s own observations of TRiP. Although the evaluator was able to observe a variety of TRiP activities and exchanges, there was a particular focus on observing the process and practices of the Intersectoral Collaboration Team and Coordinated Custom Case Conferences.


To illustrate the nature of primary data gathered through surveys, interviews, and observation, Table 7 presents the main topics of each method. Also shared in Table 7 is a description of data source and identification of specific instrument used (where applicable).

Table 7. Primary Data Topics, Instrument and Data Source by Collection Method

METHOD INSTRUMENT DATA SOURCE TOPICS

Surveys TRiP Initial Staff Survey

TRiP Staff

perspective, collaboration, benefit to clients, benefit to agencies, successes, challenges, improvements, partner participation, barriers overcome, services mobilized, service integration, key ingredients

TRiP Initial C4 Participant Survey

C4 Participants perspective, collaboration, benefit to clients, benefit to agencies, successes, challenges, improvements, partner participation, barriers overcome, services mobilized, service integration, key ingredients

Child Exit Survey 11UI clients receipt of help, help remaining, personal improvement, personal and family change

Youth Exit Survey twelve&up clients receipt of help, help remaining, personal improvement, personal and family change

Caregiver Exit Survey

caregivers of clients support provided, satisfaction, service change, service access, challenges, level of support to child, level of support to caregiver, change in child, change in parenting, service improvement

TRiP Closing C4 Participant Survey

C4 Participants perspective, collaboration, benefit to clients, benefit to agencies, successes, challenges, improvements, partner participation, barriers overcome, services mobilized, service integration, client impact

TRiP Closing Stakeholder Survey

Steering Committee Members

overall experience, collaboration, successes, challenges, improvements, client impact, agency perspective on collaboration, next steps

Interviews Staff Closing Interview Guide

TRiP Staff

accomplishment, client benefit, agency benefit, success, challenges, improvements, partner participation, barriers, service mobilization, service integration, key ingredients, overall impact, individual benefit

Observations na ICT & C4 meetings process, efficiency, collaboration, communication, challenges, improvement

5.3 EVALUATION MATRIX

To illustrate the connection between the data collection process and the main interests of an evaluation, an evaluation matrix becomes helpful. As Table 8 shows, each category within the evaluation has a number of variables. Potential indicators and methods of data collection for each variable are proposed in the matrix.


Table 8. Evaluation Matrix for 11UI and twelve&up CATEGORY VARIABLE INDICATOR COLLECTION METHOD

OUTPUTS Community Awareness

- # of presentations - audience types - information provided

- Community Outreach and Networking Form

Referrals Received - # of agency referrals - # of caregiver referrals

- Caregiver Referral Form - Agency Referral Form

Intakes Completed - # of intake interviews with caregivers - # of intake interviews with children - # of completed intakes

- Caregiver Intake Guide - Child & Youth Intake Guide

Needs Identified - # of needs by type - C4 Action Plan

Service Involvement

- # of services by service type - C4 Action Plan

Coordinated Action Plans

- # of action plans completed - C4 Action Plan

Support Provisions - # of services mobilized by type - C4 Action Plan - Case Closure Report

Removal of Barriers

- # of barriers by type - # of barriers overcome by type and strategy

- C4 Action Plan - Case Closure Report

SHORT-TERM OUTCOMES

Coordinated Service Delivery

- # of services mobilized and engaged by type

- C4 Action Plan - Case Closure Report

Children Feel Supported

- child recognition of support - parent opinions of support received by their child

- Child Exit Survey - Youth Exit Survey - Caregiver Exit Survey

Parents Feel Supported

- parent feedback on support - Caregiver Exit Survey

Barriers Reduced - # of barriers reduced by type and method of reduction

- C4 Action Plan - Case Closure Form

INTERMEDIATE OUTCOMES

Reduction in Risk - change in aggregate risk scores - Needs-Based Risk Assessment

Change in Behaviour & Attitude

- improvements in child behaviour - Case Closure Report

Increased Access to Pro-Social Activities

- participation in community activities, events and groups

- C4 Action Plan - Community Engagement Form

LONG-TERM OUTCOME

Decreased Vulnerability

- reduction in school disengagement - decrease in aggregate vulnerability

- Post-TRiP School Report - Case Closure Report

EVALUATION QUESTION THEMES

Target Group - age, gender, ethnicity - risk factors - family dynamics - service engagement - community engagement - school engagement - child development and characteristics

- School Intake Guide - Caregiver Intake Guide - Child & Youth Intake Guide


Satisfaction - child satisfaction - caregiver satisfaction - stakeholder satisfaction - case conference participant satisfaction - staff satisfaction - steering committee satisfaction

- Child Exit Survey - Youth Exit Survey - Caregiver Exit Survey - Survey to C4 Participants - Interviews with staff - Survey to steering committee

Efficiency - client access to services - service provider responsiveness

- observations of Intersectoral Collaboration Team - observations of Coordinated Custom Case Conference - Survey to C4 Participants - C4 Action Plan - Case Closure Report

Effectiveness - # of service coordination activities by type - feedback on children feeling supported - feedback on parents feeling supported - reduction in barriers by type

- Child Exit Survey - Youth Exit Survey - Caregiver Exit Survey - C4 Action Plan - C4 Action Plan - Case Closure Report


6.0 INTERIM RESULTS During the evaluation process, TRiP staff and steering committee were provided with updates on the evaluation on two occasions. The first was the Initial Evaluation Update delivered in May of 2016. The second was the Secondary Evaluation Update delivered in November of 2016. Both of these deliverables reported on the current implementation of the new internal reporting and data collection processes. The reports also highlighted some key findings and recommendations appearing in the early work of the evaluation process. The following subsections summarize results of these interim evaluation deliverables. 6.1 PRELIMINARY RESULTS To develop a preliminary understanding of TRiP’s implementation of the 11UI and twelve&up initiatives, a two-part methodology was deployed. The first part involved a quantitative case study of 40 TRiP clients (past and current). For the case study, data from the TRiP reporting tools were used to create a small database that was used to develop a basic understanding of the achieved target group and service delivery to date. The second part involved an examination of survey responses from both TRiP staff and community partners involved in the custom coordinated case conferences. During the month of April 2016, two different survey instruments were provided to the respective groups. Questions on the surveys solicited feedback on early impressions, short-term outcomes, successes, challenges and opportunities for improvement. Quantitative analysis of data from the case study revealed that TRiP had been engaging hard-to-reach clients, many of which are showing signs of anti-social behaviour, criminal involvement, mental health concerns, parenting issues and school absenteeism. Some of the services that TRiP clients have been engaged with include recreation, mental health, counselling, mentoring, and parenting support. A lot of the effort made by TRiP staff and partners has helped to reduce some of the personal, situational, financial, and institutional barriers preventing clients from getting the support that they need. Preliminary results indicate that once these barriers are reduced, a sizable number of clients are engaging in the services they require. Feedback from the staff and community partner survey indicated broad support for the initiative. Overall, most respondents felt that TRiP was a proactive and effective initiative for mobilizing multiple services around client needs. During this mobilization process, TRiP was described as providing organized and purposeful opportunities for collaboration around identifying client need and solutions to their problems. These interactions of multiple service providers have allowed different agencies to form relationships, better understand one another, and work together around a single client and their family. Several respondents felt they were generally more effective because of their collaborations with others involved in TRiP. This collaboration of multiple human service providers was described as having a positive impact on the capacity of those agencies involved. Collective knowledge of client needs and the barriers they encounter, have allowed for a direct reduction of personal, situational, institutional and financial barriers to care. According to respondents, this has had a positive impact on client outcomes. Specific examples of client outcomes, according to respondents, include improved school attendance, increased social engagement and strengthened relations with care providers.


Overall, this preliminary look at both case study and survey data allowed us to examine and better understand the effectiveness and efficiency of TRiP’s early efforts to reduce the risks affecting vulnerable children and youth. At the time, it also revealed a number of opportunities to improve the ongoing implementation of TRiP. 6.2 SECONDARY RESULTS The Secondary Evaluation Update focused exclusively on the ongoing data collection process, including the TRiP Client Database. In order to evaluate and verify that the data collection tools were being implemented and achieving the intent to which they were designed, the evaluator reviewed all files of clients who were active in TRiP on or after the launch of the new data collection process (September 2015). During this file review process, data were analysed using frequency distributions and basic categorical analysis techniques, whereas the open text entries into the database were explored through both content and thematic analysis. The main purpose of this update was to describe the state of data collection for TRiP, identify some potential trends and early observations, and offer suggestions for making improvements to both data collection practices and implementation of the TRiP model. This deliverable provided an opportunity to verify that the data collection tools being implemented, were achieving the objectives for which they were designed. The data collated and analyzed for this secondary progress update revealed that TRiP staff made a tremendous commitment to its new ongoing data collection and storage process. The results presented within the Secondary Evaluation Update began to demonstrate the achieved target group, client risk factors, service needs, collaborative efforts to support clients, and early outcomes from that collaboration. More immediately clear, however, was that the results of the report demonstrated the absolute importance of proper and complete client file reporting. Based upon the results in the report, a number of secondary findings emerged3. These include:

• Between September 2015 and November 2016, TRiP staff generated 737 documents on their client filing system.

• By September 2016, the new TRiP Client Database held data on 211 clients.

• A majority of clients were male.

• Some of the more common risk factors presented in TRiP client case files include anti-social behaviour, criminal involvement, mental health, physical violence, and parenting concerns.

• Some of the more common service needs among TRiP clients included recreation, mental health services, counselling, mentorship, and parenting support.

• The most common barriers affecting TRiP clients were financial and personal in nature.

• During the assessment process, 55 TRiP clients received a Total Needs Assessment Score. Of these, 19 scored between 6 and 19 (suggesting collaborative supports), and 36 scored over 20 (requiring custom coordinated case support).

• Most referrals to TRiP came from the education and policing sectors.

• Education was the most common lead sector on TRiP files.

• All sectors represented at TRiP assisted in case files.

3 These findings are based only upon the 211 clients with open files on or after September 2015.


• Among cases accepted to TRiP, and where data were available (N = 59), the School Engagement Officer was involved 33.9% of the time.

• The School Engagement Officer had, on average, 5 contacts with each client she was involved with.

• Within the Custom Coordinated Case Conferences, TRiP had engaged an additional 16 human service agencies to provide collaborative support to children and youth.

• At the time of this analysis, 61 client files remained open while 150 have been closed.

• Data revealed that of the 150 closed files, 4 were closed because of reduced vulnerability, 17 were closed because of potential reduced vulnerability, and the rest closed for a variety of reasons (e.g., aged out, screened out, refused consent, refused services, not engaging, in custody/care, unable to locate, moved away).

• Although very new at the time, the Post-Trip Student Report was starting to show promise in measuring improvements in education outcomes.


7.0 FINAL RESULTS Based upon data source, the results of this evaluation are divided into two separate sections. The first includes results from the analysis of data captured in TRiP’s internal reporting process for the entire evaluation period. The second set of results stems from analysis of primary qualitative data gathered through interviews, surveys, and observations made within the evaluation period. 7.1 INTERNAL REPORTING PROCESS In September of 2015, TRiP implemented a new internal reporting process designed in partnership with the evaluator. A major source of data for this evaluation are data gathered through this reporting process between September 1, 2015 and May 31, 2017. In total, 2,443 individual tools were completed by TRiP staff, clients, and caregivers. As Table 9 shows, considerably more tools were completed for the 11UI than the twelve&up program4.

Table 9. Number of TRiP Reporting Tools Filed by Reporting Tool Type and Program

Reporting Tool Type 11UI twelve&up Total

Community Networking and Outreach Form na na 130

Caregiver Referral Form 22 11 33

Agency Referral Form 223 104 327

School Background Report 64 43 107

Caregiver Intake Guide 66 43 109

Child & Youth Intake Guide 64 42 106

TRiP Risk-Based Needs Assessment 64 43 107

ICT Action Plan 69 42 111

C4 Report 877 145 1,022

Community Engagement Form 65 7 72

School Engagement Summary 9 6 15

Case Closure Report 162 81 243

Post-TRiP Student Report 50 11 61

TOTAL 1,735 578 2,443

7.1.1 Community Networking and Outreach

Although most of the reporting tools pertain to individual TRiP clients, one major area of activity that does not pertain to individual clients is community networking and outreach. Throughout the evaluation period, TRiP staff provided presentations, hosted visitors, sought funding, and engaged in dialogue around TRiP processes with multiple audiences. To provide some understanding of the nature, type, and quantity of TRiP networking and outreach, Table 10 summarizes data captured using the Community Networking and Outreach Form. As shown in Table 10, most presentations were co-delivered by the TRiP Coordinator and Initiative Strategist (n = 69). A majority (60%) of meetings were initiated by the other party (not TRiP). There was a general balance in the format of presentations between formal (n = 39), casual (n = 36), or other (n = 45). The participant types most often engaged included funders (n = 33) and existing partners (n = 35),

4 The reason for this imbalance is because 11UI was implemented sooner than twelve&up and has been able to accumulate more client files.


followed by local organizations (n = 23) and potential partners (n = 15). While most interactions involved 1 to 3 participants (n = 75) (in addition to TRiP staff), some larger presentations involved over 500 participants. In general, TRiP staff reported a majority of reactions from participants to be positive (n = 111). Finally, of the 37 interactions that concluded with a recommendation, most ended with a call for further meetings (n = 19).

Table 10. Summary of Community Networking and Outreach Interactions (N = 130)

Variable Variant N %

Presenter coordinator initiative strategist both other staff all of above

16 1

69 34 10

12.3 0.77 53.1 26.1 7.7

Initiator we asked they asked conference other undetermined

40 78 1 3 8

30.8 60.0 0.77 2.3 6.2

Format formal casual sharing other undetermined

39 36 8

45 2

30.0 27.7 6.2

34.6 1.5

Participant Type funder existing partner potential partner local organizations different community media political leader missing

33 35 15 23 13 4 5 2

25.3 26.9 11.5 17.7 10.0 3.1 3.8 1.5

Number of Participants

1 – 3 4 – 15 16 – 49 50+ missing

75 27 7 5

16

57.7 20.8 5.4 3.8

12.3

Reaction positive neutral negative unsure missing

111 7 5 0 7

85.4 5.4 3.8 0.0 5.4

Recommendation keep in touch more meetings follow-up soon specific action nothing

8 19 3 7

93

6.2 14.6 2.3 5.4

71.5


7.1.2 Referrals As each form is completed and submitted to the Administrative Support Specialist, data from the forms are entered onto the TRiP Client Database. Within the evaluation period, data had been captured on 360 clients referred to TRiP. As of May 31, 2017, 335 clients were invited to join TRiP, while 25 referrals were screened out (rejected). As Table 11 shows, a majority of referrals came from the education sector, followed by policing and health.

Table 11. Referral Source (N = 360)

Referral Source N %

Catholic Education 48 13.3

Public Education 133 36.9

Justice 22 6.1

Police 70 19.4

Social Services 10 2.8

Health 37 10.3

Caregiver 33 12.7

Outside Agency 7 1.9

7.1.3 File Status

One of key variables captured in the TRiP Client Database is File Status. This allows both the evaluator and TRiP Coordinator to monitor overall caseload and current demand placed upon TRiP staff. As of May 31, 2017, 100 client files remained active, while 259 files had been closed (see Table 12).

Table 12. File Status as of May 31, 2017 (N = 360)

File Status N %

Active 100 27.8

Closed 259 71.9

Wait-List 1 0.3

7.1.4 Achieved Target Group

During the evaluation period, TRiP was able to engage a target group that was largely male (78.8%). The majority of client’s ages varied between 5 and 14 years of age (see Table 13).

Table 13. TRiP Client Demographics (N = 360)


Gender male 284 78.8

females 76 21.1

Age (at time of referral)

5 to 8 111 30.8

9 to 11 120 33.3

12 to 14 97 26.9

15 or older 32 8.8


One of the important roles of the TRiP assessment and intake process is to identify the different risk categories affecting a client. This helps to determine the needs of clients and begin engaging appropriate supports in the Custom Coordinated Case Conference. As Table 14 shows, some of the more common risk categories affecting TRiP clients include anti-social behaviour, criminal involvement, mental health, physical violence, and parenting concerns, among others. When reading Table 14, it is important to note that it is very common for clients of TRiP to present multiple risks.

Table 14. Risk Categories Affecting TRiP Clients (N = 301)*

Risk Category N of Clients Presenting Risk Category

alcohol 68

anti-social behaviour 185

basic needs 50

crime victimization 65

criminal involvement 132

cognitive impairment 55

drugs 79

elderly abuse 3

emotional violence - perpetrator 88

emotional violence - victim 80

gambling 0

gangs 42

housing 24

mental health 178

missing school 121

missing/runaway 55

negative peers 101

parenting 183

physical health 22

physical violence - perpetrator 117

physical violence - victim 92

poverty 74

self-harm 66

sexual violence - perpetrator 18

sexual violence - victim 27

social environment 85

suicide 50

supervision 101

threat to public safety 60

unemployment 35

* Data on risk were available for 301 clients.

Once client needs assessments are complete, a determination is made whether to “accept”, “close” or “waitlist” the client. If accepted, TRiP’s Intersectoral Collaboration Team (ICT) works to identify the appropriate services that would help meet the needs of clients, and ultimately, reduce their vulnerability. Results of the analysis show that some of the more common services suggested for clients include recreation, mental health, counselling, mentorship, and parenting support (see Table 15).


Table 15. Service Needs of TRiP Clients (N = 188)*

Service Needs N of Clients Presenting Service Need

social services 27

social assistance 2

housing 12

mental health 126

sexual health 9

public health 6

medical health 8

addictions 16

harm reduction 9

counselling 98

cultural support 34

spiritual support 14

parenting support 137

education support 60

employment support 11

home care 0

life skills 43

victim support 27

safe shelter 2

police 10

courts 4

corrections 5

probation 0

parole 0

legal support 1

fire department 18

mentorship 98

recreation 151

food support 2

other 14

* Data on service needs were available for 188 clients.

During the assessment process, a number of barriers to support and services are identified. This helps the Custom Coordinated Case Conference participants have a better understanding of ways in which they can implement effective supports for the client. Within the database, some of the more common barriers were financial and personal barriers to support. As Table 16 shows, some of the examples for personal barriers include distrust with service providers, attitude towards help, cognitive delay, and lack of commitment. Some of the financial barriers include inability to afford certain activities and lack of transportation to supports and services.


Table 16. Barriers Affecting TRiP Clients (N = 190)*

Barrier Type N Examples

personal 154 distrust, attitude, cognitive delay, lack of commitment

financial 147 cannot afford activities, lack of transportation

situational 20 parents not supportive, family conflict, isolated, challenging neighbourhood

institutional 10 screening thresholds, gaps in service

multiple barriers 122 mix of the above * Data on barriers were available for 190 clients.

One of the most critical tools in TRiP’s new data collection process is the Risk-Based Needs Assessment. This instrument requires TRiP staff to collate data from other TRiP reporting tools into categories of risk, life experience, community engagement, school engagement and school absence. A built-in weighting scheme allows TRiP staff to generate a total needs score of clients. In interpreting these scores, a score of 0 to 5 suggests the client could benefit from support, and that the client may be connected to supports outside of TRiP—but a full coordination of supports is not necessary. A score between 6 to 19 suggests the client would likely benefit from support and that the Intersectoral Collaboration Team could explore options to support the client. A score of 20 or higher indicates that the client is in significant need of support and that TRiP should prepare for a Custom Coordinated Case Conference. As Table 17 illustrates, a majority (70.1%) of the clients that were assessed using the Risk-Based Needs Assessment, scored 20 or higher.

Table 17. Number of TRiP Clients by Total Needs Assessment Score Groupings (N = 107)*

Needs Score Suggested Actions N %^

0 to 5 connect to appropriate unilateral supports 0 0

6 to 19 explore potential collaborative supports 32 29.9

20 or higher prepare for custom coordinated case conference 75 70.1

not scored client not assessed (e.g., refused consent, screened out, old intake process) 253 - * Data on Total Needs Assessment Score were available for 107 clients. ^ Percent based off all clients scored (n = 107).

7.1.5 Sector Involvement

Foundational to TRiP’s mandate is offering clients support and services from a collective of multi-sector professionals. From the point of referral to closure, there are several different points at which a particular human service sector can become involved in a client’s file. The TRiP Client Database captures data on sector leads, involved sectors, and Custom Coordinated Case Conference participation. These data are important not only to show the access and exposure clients are getting to different human service sectors, but to also show the different roles that each sector plays in the TRiP initiative. Once a client’s needs are assessed, an appropriate sector is identified as the “lead” in the client file. Within the evaluation period, representatives from the two school divisions in Regina are most often the lead in TRiP client files. Although one sector remains the lead, other sectors represented on the TRiP team also play a role in coordinating and mobilizing a multi-sector group of professionals to support that client. As Table 18 shows, most of the TRiP sectors have all played a role in assisting with TRiP cases.


Table 18. Number of TRiP Case Sector Leads and TRiP Assisting Sectors by Type

Variable Sector^ n %*

TRiP Case Sector Lead Among Accepted Case Files (N = 196)

public education 128 65.3

catholic education 22 11.2

social services 20 10.2

mental health 16 8.2

police 1 0.5

justice 9 4.6

TRiP Case Assisting Sector Among Accepted Case Files (N = 192)

public education 149 77.6

catholic education 31 16.1

social services 39 20.3

mental health 58 30.2

police 11 5.7

justice 10 5.2 * Percent calculated using column n divided into total data N. ^ Cases that are closed, rejected, or in the referral stage do not have sector roles assigned.

Once a client’s needs, suggested services and barriers are identified, TRiP engages a number of partner agencies in an ongoing Custom Coordinated Case Conference. These agencies are mobilized to provide services and participate in coordination of support for each TRiP client. To illustrate the diversity of services engaged in C4 meetings, Table 19 shares 40 randomly-selected organizations who have participated in C4 meetings organized by TRiP.


Table 19. Examples of Custom Coordinated Case Conference Agencies/Professions Involved in TRiP

Agency

Aboriginal Advocate Program - Regina Public School Division

Aboriginal Family Service Centre

Addiction Services - Regina Qu’Appelle Health Region

Alcohol and Drug Services - Regina Qu’Appelle Health Region

Catholic Family Services Regina

Artragious (art therapy)

Autism Resource Centre

Big Brother Big Sister of Regina and Area

Catholic Family Services Regina

Child and Youth - Regina Qu’Appelle Health Region

Cognitive Disability Strategy

daycares

Regina Dream Brokers - Sask Sport

Elders

Ezekiel Homes - Regina Youth for Christ

Family Treatment Program - Ranch Ehrlo Society

First Nations University Mentor Program

Foxvalley Counselling Services Inc.

KidsFirst Regina

learning resource teachers

Mental Health - Regina Qu’Appelle Health Region

Ministry of Social Services

occupational therapists

psychologists

Ranch Ehrlo Society

Randall Kinship Centre - Regina Qu’Appelle Health Region

Regina Catholic School Division

Regina Fire Department

Regina Open Door Society

Regina Police Service

Regina Public School Division

Regina Qu’Appelle Health Region

Regina Open Door Society

Socialization, Communication and Education Program

Structured Learning Program - Public & Catholic School Divisions

Speech Pathologist - Public & Catholic School Divisions

School Resource Officer - Regina Police Service

Settlement Workers in Schools Sask - Regina Open Door Society

Thomas Circle of Care Inc.

Regina Open Door Services

7.1.6 School Engagement One important component of TRiP is the School Engagement Officer seconded to TRiP on a full-time basis from Regina Police Service. The role of this position is to assist in removing the barriers to student attendance and engagement in education. Within the TRiP Client Database, among the 360 cases accepted to TRiP (active, waiting or closed—not including ‘rejected’ cases) during the evaluation period,


the School Engagement Officer was involved in at least 21.4% (n = 77) of the cases (see Table 20). However, when narrowing the analysis to clients who were considered to be “engaged” in TRiP, this proportion increases to 37.8% (n = 56)5.

Table 20. Number of Accepted Cases Involving School Engagement Officer

All Clients (n = 360)

Engaged Clients (n = 148)

SEO Involvement N % N %

Yes 77 21.4 56 37.8

No 283 78.6 92 62.2

Within each school engagement with a TRiP client, the School Engagement Officer would connect with that client anywhere from 1 or 2 times, to upwards of 57 different times. On average, the School Engagement Officer has met with each student approximately 5 times6. 7.1.7 Community Engagement Another area of support for TRiP clients is community engagement. Led by TRiP’s part-time support workers, community engagement involves supporting families in accessing services, recreation, and community activities; reducing barriers; and providing positive mentoring and support. During the evaluation period, 72 Community Engagement Forms were completed. Data from these forms indicate the occurrence of 98 individual deliveries of community engagement support to 33 different TRiP clients. While some clients (n = 13) engaged in multiple activities, most (n = 20) engaged in a single activity. Data from the Community Engagement Form indicate that most of the activities facilitated by support workers can be grouped into 6 different categories. These include sports, educational, retail, event, food, and entertainment. As Table 21 shows, the two most common activities included sports (n = 44) and educational (n = 24) activities. Results from the analysis show that most activities (98%) were planned or suggested by TRiP staff. Overall, a majority (56%) of clients showed a positive reaction (e.g., engaging, excited, sociable) to the activity. Others showed a neutral (10.2%) or negative (18.4%) reaction to the activity. With respect to duration of activities, some were one-time activities (n = 22) while others were occasional (n = 14), or ongoing (n = 18). According to submitted data, support workers provided transportation 38 different times. Finally, some of the main barriers overcome in pursuit of the activity include personal (25.0%), financial (35.7%), and situational (35.7%) barriers.

5 For the purposes of this analysis, a client was “engaged” in TRiP if they had either received support through a Custom Coordinated Case

Conference, or received direct service mobilization through their initial Intersectoral Collaboration Team meeting. 6 For a more comprehensive examination of the School Engagement Officer position, see the Special Analysis in Section 8.0 of this report.


Table 21. Summary of Community Engagement Form Results

Variable Variant N %*

Activity sports 44 44.8

educational 24 24.5

eating 6 6.1

entertainment 8 8.2

retail 3 3.1

event 13 13.3

Origin of Plan client identified interest 1 2.0

staff suggested activity 49 98.0

parent suggested activity 0 -

missing 48 -

Client Reactions positive 55 56.1

neutral 10 10.2

negative 18 18.4

unsure 15 15.3

Duration of Activity

one-time 22 40.7

occasional 14 30.0

ongoing 18 33.3

missing 44 -

Transport Provided

yes 38 42.2

no 52 57.8

missing 8 -

Barriers to Activity

personal 7 25.0

financial 10 35.7

situational 10 35.7

institutional 1 3.8

other 0 -

* % values do not include missing data in quotient.

7.1.8 Case Duration

Once accepted into the 11UI or twelve&up initiative, TRiP clients spend anywhere from a few days to several years receiving coordinated support. To gauge the different durations of TRiP client support, the number of months between the referral date and closure date were calculated using data from the TRiP Client Database. As Table 22 illustrates, a larger proportion of clients receive support for 5 months or less. Further analysis of reporting data showed no relationship between duration of support and client outcomes, nor between duration of support and reason for closure.

Table 22. Duration of TRiP Client Support by Duration Length (N = 254)

Duration N %

Less than 1 month 88 34.6

Between 1 month to 5 months 78 30.7

Between 6 months to 12 months 38 15.0

1 year or more 50 19.7


7.1.9 Case Closure As the TRiP team wraps up its support with vulnerable children and youth, client files are closed for a variety of reasons. Table 23 shows that of the 243 client files with a completed Case Closure Report, 14.8% (n = 36) of these files have been closed because the client moved away or was unable to locate7. In contrast, at least 23.9% (n = 58) of client files were closed because the client had either “stabilized” or had the “potential to stabilize”. Additional reasons for closure included “refused services” (12.7%), “went into custody” (6.9%), or “did not engage” (27.1%).

Table 23. Reason for TRiP Case Closure (N = 243)

Reason for Closure N %

In Custody 17 7.0

Moved Away 23 9.5

Did Not Engage 66 27.1

Refused Consent 2 0.8

Refused Services 31 12.8

Unable to Locate 13 5.3

Potential to Stabilize 46 18.9

Stabilized 12 4.9

other 33 13.6

7.1.10 Client Outcomes

As case files become closed, TRiP staff record information using the Case Closure Report. Qualitative data from the report were coded into binary data (yes/no), then transferred to the TRiP Client Database. Results from the analysis of these data provided information in a number of areas. As Table 24 shows, at the time of closure, slightly over half (58.6%) of all clients [with available data] were connected to services during their individual TRiP support period. Similarly, 76.7% (n = 109) of clients were able to overcome barriers to service. With respect to community activities, a majority of closed TRiP client files show that clients were able to engage in (48.8%), and overcome barriers to (75.4%) pro-social community activities. In addition to supporting clients, results of Case Closure Report data show that 47.6% (n = 119) of families also received services and supports from TRiP. Finally, one of the last sections in the Case Closure Report records staff, care provider, and human service partner observations of reduced vulnerability among TRiP clients. According to these results, 65.9% (n = 56) of 85 clients with available data displayed an observable reduction in vulnerability.

7 Does not include client files rejected during the referral/intake process (n = 25).


Table 24. Client Outcomes at Time of Closure

Outcomes Yes ^ No ^ missing

N % N % N

Connected to Services (n = 278)* 163 58.6 115 42.4 0

Service Barriers Overcome (n = 142) 109 76.7 33 23.2 113

Engaged in Activities (n = 252) 123 48.8 129 51.2 3

Activity Barriers Overcome (n = 142) 107 75.4 35 24.6 113

Family Concerns Overcome (n = 98) 53 54.1 45 45.9 157

Services Provided to Family (n = 250) 119 47.6 131 52.4 5

Positive Change Behaviour (n = 91) 57 62.6 34 37.4 164

Observable Reduction in Vulnerability (n = 85) 56 65.9 29 34.1 170

* In addition to the Case Closure Report, data from other reporting tools were used to measure Connected to Services. ^ Column percentages represent percentage of available data for each outcome.

To further explore client outcomes in these areas, a proxy for Aggregate Reduction in Vulnerability was created. First, a client sample (n = 148) was identified to include clients who were considered to have “engaged” in TRiP. Next, outcome data were recoded into a value of 1 point per recorded outcome (e.g. connected to services, family concerns overcome). Following the data recoding, all 8 outcome variable values were summed to generate a total score, with 0 being the lowest possible score and 8 being the highest possible score. The average score within the sample was 5.2. As Figure 3 shows, when broken into categories, 27 clients showed some reduction in vulnerability (score: 1 – 2), 60 clients showed a moderate reduction in vulnerability (score: 3 – 5), and 61 clients showed a strong reduction in vulnerability (score: 6 – 8). None of the 148 clients scored 0 (i.e., no reduction). Overall, these results suggest that among clients who “engaged” in TRiP, a majority (81%) experienced a moderate or strong reduction in vulnerability due to an aggregate impact of individual-level outcomes.

Figure 3. Results of Aggregate Reduction in Vulnerability (n = 148)

18%

41%

41%

Some Reduction Moderate Reduction Strong Reduction


Another method used to track client outcomes is an examination of data gathered from the Post-TRiP School Report. This report, completed by school staff, gathers data on student attendance, student behaviour and student performance. As Table 25 illustrates, of the 53 clients who had completed attendance data in the Post-TRiP School Report in their file (as of May 31, 2017), the school attendance of 50 (94.3%) clients either “stayed the same”, “improved”, or was considered “good”. In contrast, only 3 (5.7%) clients had “poor” or “sporadic” attendance following TRiP support.

Table 25. Post-TRiP School Attendance (n = 53)*

School Attendance N %

Poor 2 3.8

Same 16 30.2

Sporadic 1 1.9

Improved 15 28.3

Good 19 35.8 *n = number of client files closed and rated on school attendance.

Regarding school behaviour following TRiP support, 13 (23.2%) clients were observed to have “improved” their behaviour, 10 (17.9%) stayed the “same”, 18 (32.1%) were considered “good”, and only 4 (7.1%) were considered to have “poor behaviour” following TRiP support (see Table 26).

Table 26. Post-Support School Behaviour (n = 56)*

Behaviour N %

Poor 4 7.1

Same 10 17.9

Average 11 19.6

Improved 13 23.2

Good 18 32.1 *n = number of client files closed and rated on school behaviour.

With respect to overall school performance among clients supported by TRiP, school staff examined both strength in literacy and in numeracy. Results of the analysis reveal that 44 (86.2%) of 51 clients fall between “beginning” or are currently demonstrating “good” performance in school. In contrast, 7 (13.7%) clients demonstrate “insufficient” performance or “no change” in performance (see Table 27).

Table 27. Post-Support School Performance (n = 51)*

Performance N %

Insufficient 1 2.0

Same 6 11.8

Beginning 8 15.7

Progressing 15 29.4

Improved 4 7.8

Meeting 15 29.4

Good 2 3.9 *N = number of client files closed and rated on school performance.


7.2 PRIMARY DATA RESULTS To gather rich descriptive accounts of the factors being explored in this evaluation, open-ended surveys were given to child participants, youth participants, and caregivers. These instruments were provided to respondents near the end of their individual support periods with TRiP. In addition, open-ended surveys were administered to both C4 participants and steering committee members in March of 2017. Also at this time, TRiP staff were interviewed in-person by the evaluator8. As Table 28 shows, 22 child clients, 7 youth clients, 41 caregivers, 5 steering committee members, 10 C4 participants, and 11 TRiP staff provided feedback to this evaluation process.

Table 28. Number of Primary Data Collection Instruments Completed by Type

Reporting Tool Type Total

Child Exit Survey 22

Youth Exit Survey 7

Caregiver Exit Survey 41

C4 Participant Survey 10

Steering Committee Member Survey 5

Interviews with TRiP Staff 11

TOTAL 97

Data from the surveys and interview process were reviewed and analysed using thematic content analysis. The following sub-sections summarize the feedback given by each of the respondent groups. 7.2.1 Child Exit Survey Feedback In responding to the Child Exit Survey, all 22 respondents felt that they had been helped by TRiP. Some identified particular staff and human service providers that were helpful, while others named activities and particular experiences they found to be helpful. When asked if they felt they still needed help, 14 identified different types of additional help that they wanted. The remaining 6 respondents expressed that they did not feel any additional help was necessary. The third question on the survey asked respondents to identify how they may have become better through their experience with TRiP. As Table 29 shows, a variety of responses were provided. Some of the ways respondents report they have become better include, self-monitoring, personal relationships, skills, and overall behaviour. The final question asked of child respondents was whether there was anything different about their family. Responses suggest that improved relationships, increased activities, and more stable family dynamics resulted from participation.

8 For the purposes of this report, TRiP staff includes 2 school liaisons, 1 coordinator, 1 initiative strategist, 1 case manager, 1 school engagement

officer, 1 referral officer, 3 sector representatives, and 1 administrative specialist.


Table 29. Child Exit Survey - Summary of Respondent Feedback

Question Dialogue

Has anyone helped you?

counsellor ● dream broker ● TRiP staff ● Elder ● big brother volunteer ● teachers ● social workers ● school liaison ● principal ● parents ● community school coordinator ● aunt ● grandparents ● parkour instructor ● school staff ● boxing coach

Do you still need help?

want to swim more ● still get frustrated ● long waitlist for Big Brothers ● more activities ● want to play hockey ● help with organizing ● need to learn to take turns ● need to stop using bad words ● need help with my anger ● I still get confused ● always room for improvement ● need more motivation

How have you become better?

happier ● better in school ● better coordination of my day ● better on trampoline ● getting school work done ● new friends ● talk to others easier now ● I ignore bullies ● I don’t run away anymore ● involved in activities ● make better choices ● tell the truth ● listen to adults more ● make family happier ● better at reading and math ● new responsibilities ● trying harder in school ● learned to talk about my problems ● better hockey player ● doing more homework ● new interests ● don’t get angry anymore ● more active ● less video games ● less stress ● better body language ● I try more things now ● I want to attend school ● I get out of bed in morning now ● I now know how to ‘let it go’ ● have fixed my temper ● less angry ● show more respect to adults

Is there anything different about your family?

we have more fun at home ● go swimming ● do more activities ● spend more time with mom ● hang out together more ● go to museums ● mom is happier ● dad went into treatment ● better routines ● siblings are in school now ● get along more because I am behaving ● mom makes me do more ● we don’t fight as much now

7.2.2 Youth Exit Survey Feedback Among the few respondents (n = 7) who completed the Youth Exit Survey, all reported that participating in TRiP was helpful. When asked whether anyone had helped them lately, respondents identified a variety of human service professionals, family members, and community assets. In describing remaining needs, some felt they could use additional supports—for things such as boredom, anger, and family-related issues. The question on what each respondent was able to improve revealed changes in behavior, improvements in relationships, and improved problem-solving. Finally, respondents who completed the Youth Exit Survey identified a number of changes in their family since receiving support from TRiP. As Table 30 shows, some of these include improved family communication, increased pro-social activities, and positive feelings of support.

Table 30. Youth Exit Survey - Summary of Respondent Feedback

Question Dialogue

Has anyone helped you lately?

police ● mother ● teachers ● principal ● school ● archer ● dream brokers ● support workers ● social worker ● TRiP staff ● music teacher ● counsellor ● health region


I still get frustrated at times ● need to join more sports ● still some anger in home ● no I’m good now ● not really ● I’m ok ● nope

What have you been able to improve lately?

less fighting ● better choices ● I tell my mom stuff ● try to be involved more ● coordination ● trampoline skills ● my reading ● including others ● control anger ● show more respect for my mother ● better relationships with my classmates ● better temper ● socialize with others ● react to people better

Is there anything different about your family?

able to talk more ● less fighting in the home ● swimming ● spend more time together ● feel really included ● we talk more as a family ● easier to talk to my mom


7.2.3 Caregiver Exit Survey Feedback One of the most information-rich tools for data collection in this evaluation is the Caregiver Exit Survey. Administered to caregivers following exit from the program, the tool gave a deep understanding of the family experience, reaction, benefits, and challenges during their involvement in TRiP. In total, 41 caregivers responded to the survey. The first question asked of respondents inquired about the support provided to their child and family from TRiP. Feedback indicated that TRiP provided support in four key areas. These include providing individual support to the family, access to services, engagement in activities, and reduction to barriers. Some of the feedback on how TRiP provided individual support showed a strong appreciation for the team approach that TRiP supports. For example, one caregiver shared that “having everyone on the same page was very beneficial”. Another felt that “knowing everyone was on the same team to help was so important for my child”. Other efforts appreciated by caregivers include open lines of communication with TRiP staff, regular team meetings on child progress, strong moral support, facilitated case conferences, support in engaging human service partners, and as one caregiver described, “just having someone there that knows what she’s going through is important.” When it came to service access, several caregivers explained that TRiP helped their child(ren) and family access services they wouldn’t ordinarily have accessed. Some of these services include parenting support, counselling, school support, mental health services, emotional care, behavior programs, and community support. Similarly, TRiP also helped clients engage in recreational activities they wouldn’t have had the opportunity to engage in. Some of the activities mentioned by caregivers include swimming lessons, music lessons, science centre visits, leadership luncheons, martial arts, various sports, camping, parkour, and biking. The second question on the survey asked respondents how satisfied they were with the overall delivery of support through TRiP. As demonstrated in Table 31, a majority of respondents (85.4%) were either ‘very satisfied’ or ‘satisfied’.

Table 31. Caregiver Exit Survey – Respondent Satisfaction with TRiP Delivery of Support (N = 41)

Level of Satisfaction N %

Very Satisfied 23 56.1

Satisfied 12 29.3

Somewhat Satisfied 5 12.2

Not Satisfied 1 2.4

The third question asked of respondents whether they had previously accessed human service supports. This question was designed to generate some understanding of past client experience with human service professionals. In total 33 (80.5%) of respondents had some previous engagement with human service professionals. A follow-up to this question was whether they saw a noticeable difference in the way they received services through TRiP’s coordination versus previous single sector support. All 33 respondents identified that indeed, the service delivery they encountered through TRiP was much different than their previous experiences. Some of the differences mentioned by respondents are summarized in Figure 4.


Figure 4. Caregiver Exit Survey – Differences Between TRiP and Past Service Delivery

SUMMARY OF DIFFERENCES

• TRiP made me feel more supported.

• Things seemed a lot more stable and organized than usual.

• We got access to services much quicker this way.

• We were not making a lot of headway until TRiP became involved.

• Having a group support us got questions answered quickly.

• There was much more consistency in the care we received.

• The setting seemed a lot friendlier and less judgmental.

• TRiP seemed a lot more hands-on and easier to connect with workers.

• There is much more support in this approach compared to when they work alone.

• TRiP got us meetings with people we could never get to before.

• It was nice not to have to start over with every new face. TRiP kept them informed.

• It was much better to have access to everyone at once.

• There seemed to be more action, rather than just sitting there talking to a counsellor.

• It wasn’t so clinical, it seemed more supportive, friendly, playful, and positive.

The next topic explored with caregivers was whether TRiP made it easier for caregivers and their children to access services. All 41 respondents felt that TRiP had helped to improve service access. Many of the respondents explained that TRiP had informed them of services that they had no knowledge of previously. Others shared that TRiP staff were very effective at not only informing families of services, but in helping them to connect and engage in these services. Caregivers identified that art of the reason for these successful engagements was that TRiP staff were available when families needed them. As one respondent shared, “being able to get a hold of [TRiP] when we needed them made a world of difference for trying to manage our help.” According to respondents, other ways in which TRiP helped to improve client access to services involved taking care of some important tasks and activities. These include:

• transportation

• registration

• logistics

• flexibility

• coordination of services

• communication among partners

• equipment

• proper referrals Finally, several respondents felt that TRiP staff made the entire process smooth by being so helpful, consistent, and supportive. Several commented on the convenience of being able to see multiple service providers at once. This not only helped to secure initial service access, but was the impetus for further service delivery beyond TRiP’s initial work. In contrast to the previous question, the next question asked respondents whether anything made it difficult to access services. Among all 41 respondents, only 9 respondents pointed out existing barriers to service. The only TRiP-related barrier mentioned from one respondent was that there were a lot of


meetings. Other remaining barriers to service included personal illness, low income, changes in the service agencies, complicated family schedules, long waitlists, and summer holidays of human service providers. Two isolated challenges mentioned in the survey included one respondent who didn’t have a good understanding of what TRiP was, and another respondent who had two sets of meetings because her children attended two different schools. The next question on the survey asked respondents to identify how supported they believed their child felt in the program. Similarly, the 7th question in the survey asked respondents to rate how supported they themselves felt. As Table 32 shows, both caregivers (82.9%) and their children (75.6%) were reported by respondents to have felt ‘very supported’ or ‘supported’ in the program.

Table 32. Caregiver Exit Survey – Caregiver/Client Feelings of Support (N = 41)

Level of Support Caregiver Feeling Child Feeling

N % N %

Very Supported 25 60.9 21 51.2

Supported 9 22.0 10 24.4

Somewhat Supported 6 14.6 7 17.7

Not Supported 1 2.4 1 2.4

Unsure 0 - 2 4.9

To further understand how TRiP may have helped families become involved in any activities, supports, or experiences that they had not enjoyed previously, respondents were asked to identify whether TRiP allowed them to do anything they were unable to do before. Of 41 respondents, 28 (68.3%) indicated they had, whereas 11 (26.8%) felt they had not (see Table 33). Only 2 respondents (4.9%) had reported that they were unsure. In providing further dialogue to this question, caregivers identified a variety of activities they would not have otherwise been able to engage their children in. These include music lessons, recreation, summer camp, hockey, shopping, swimming, acquiring a pet, and martial arts, among others. Some of the ways caregivers described why TRiP helped secure these opportunities include helping families overcome financial barriers, transportation challenges, and a lack of knowledge on available opportunities.

Table 33. Caregiver Exit Survey – TRiP Support in New Activities/Opportunities

Has TRiP allowed your family to do anything you were unable to do before?

Caregiver Feeling

N %

Yes 28 68.3

No 11 26.8

Unsure 2 4.9

One of the most important questions explored in this evaluation has been the impact of TRiP on clients of the initiative. To gather a caregiver perspective, respondents were asked whether they had noticed a change in their child since their involvement with TRiP. Overwhelmingly, 90% (n = 37) of caregivers noticed a change in their child. To provide detail around some of these changes, Figure 5 provides a summary of caregiver explanations for how their child has changed since being supported by TRiP.


Figure 5. Caregiver Exit Survey – Observed Changes in Children

SUMMARY OF CARE-GIVER REPORTED DIFFERENCES

• Less meltdowns and improved behaviour at both home and school.

• Increased respect for others.

• Improved relationships with professionals.

• Increased self-control.

• Better ability to communicate effectively.

• Increased confidence.

• Improved listening skills.

• Reduced outbursts and temperamental behaviour.

• Better management of feelings.

• Improved impulse control.

• Self-initiative to join activities.

• Fewer disruptions in class.

• Improved school attendance.

• Increased stability.

• Ability to role model.

• More sociable attitude.

• Self-growth and maturation.

• Reduction in abusive behaviour.

• Increased respect for others.

• Better understanding of acceptable behaviour.

• Improved ability to confide with adults and ask for help.

• Increased value of things (e.g. bedroom, family).

• More relaxed in group settings.

• More comfortable talking with teachers and adults.

One particular observation shared by a caregiver respondent suggested that although TRiP may have had an impact on her child’s behaviour, so did the removal of support when TRiP closed her child’s file. According to the caregiver, “since our support [from TRiP] ended, he got involved with the wrong crowd and things have gotten worse.” A related question on behaviour was then asked of caregivers. In particular, respondents were asked whether they had noticed any changes in their own parenting because of TRiP. Overall, 25 (61%) respondents indicated that they had noticed a change while 16 (39%) reported no change in their parenting that may be attributable to TRiP. To summarize respondent feedback to this question, Figure 6 provides the comments gathered through the Caregiver Exit Survey.


Figure 6. Caregiver Exit Survey – Observed Changes in Parenting

SUMMARY OF SELF-REPORTED CHANGES IN PARENTING

• I can now carry through with disciplinary action.

• It has given me more strength to have a voice in things.

• I have new ideas on how to talk to my child.

• I parent differently now, I do not internalize things and I talk to my kids.

• We listen more and respect one another’s opinions.

• I check with my child more often to see how they are feeling.

• I am trying to be more patient and understand my child better.

• We have a more structured environment in our home.

• I have started to ask for advice on how to answer situations.

• I am working on trying new things as a parent.

• I have tried new approaches on how to deal with things at home.

• I am now taking away privileges when behaviour is bad.

• I find when there is tension I am a lot calmer.

• I am spending more time with my family…thanks to TRiP, I didn’t have to work overtime to pay for these activities.

• We have two independent households, yet now have a consistent approach to parenting.

• We can talk to each other better now.

• We have structure in our home that we didn’t have before.

The next question on the survey asked respondents to identify whether they had any suggestions for improving TRiP. In general, most caregivers felt that TRiP was relatively strong and did not need any improvements. Even so, several respondents were able to offer a few suggestions for improving the initiatives. Some of these suggestions regard adjustments to current service, such as: more frequent contact; backup supports when the lead is away; longer time periods of support; and not ending all service supports and services at once when the file closes. Other suggestions for improvement concern new or expanding components to TRiP. These include: group sessions with parents; transportation on evenings and weekends; development of stability plans before file closure; and fewer meetings but more individual mentoring and family support. Among respondent suggestions for improvement, one caregiver respondent shared her experience on how file closure coupled with changes in partner agency staff, had negatively impacted her family’s experience with TRiP. According to the respondent:

Since the TRiP support ended, my child’s regular worker has moved away and there has been no communication with anyone to see how my child is doing. TRiP should make sure proper supports are in place before discharging, and when workers change at the agencies, we should get communication from new workers.

The final question asked of respondents, inquired as to whether they had any additional comments on their experience with TRiP. To preserve the original intent of this feedback, Figure 7 provides the direct comments from caregiver respondents.


Figure 7. Caregiver Exit Survey – Additional Comments on TRiP

SUMMARY OF ADDITIONAL CAREGIVER COMMENTS ON TRIP

• They were really great.

• Keep up the good work.

• It’s a good program and it has come a long way since my older son was involved.

• We met some good friends through these programs.

• TRiP has given me the education and support to help my son…thanks!

• This program has really helped our family overcome some things.

• The program was really good. The staff were so helpful.

• I was really glad to be part of this program.

• It made a big difference in getting my children to improve their life.

• I just wish TRiP could have got involved in helping my child sooner.

• There are great people working in this program.

• The program is a huge benefit to kids who are having difficulties.

• It really helped out my child drastically.

• It was really easy to get a hold of TRiP and access their staff.

• The staff were really good to work with.

• They made my family feel comfortable.

• It really saved my life and I have lots of good things to say about it.

• It was an amazing experience, my child loves TRiP.

• TRiP has been so good to our family that I want to cry every time I think about it.

• I am grateful that TRiP is ready to reengage if my son starts to slip again.

• Knowing there are supports available is really helpful and knowing that people understand what you’re going through helps tremendously.

• It gave the kids good opportunities.

• TRiP staff was persistent in fighting to get the supports our child needed, we would have never succeeded on our own.

• Our family has benefited so much by working with TRiP…we were very emotional when we learned TRiP had to move on and help other families.

• TRiP helped us build a relationship with teachers and the principal.

• It made a huge change in our family.

• We were totally lost without TRiP and had no place to go….thank you!

• Having a team support us made things so much better for us.

7.2.4 Steering Committee Survey Feedback The survey administered to 5 steering committee members solicited feedback on their overall experience with TRiP, benefits of TRiP to agencies, successes, challenges, suggestions for improvement, impact on clients, impact on collaboration, and next steps. Results on these topics are shared in the following sub-sections.

The TRiP Experience The first question in the survey asked steering committee members to describe their experience with TRiP. Overall, there was considerable satisfaction among the different steering committee members. Feedback from respondents described their experience with TRiP as collaborative, positive, validating, and beneficial to their organizations. Respondents felt that TRiP was important to clients, valuable to the community, and an exemplary model within the human service field. In fact, during this opening


conversation with steering committee members, TRiP was described as an “evolutionary”, “evidence-informed”, and a “model for the future of community safety and well-being in Regina.” Several steering committee members also shared their belief that the model could and should be replicable elsewhere. Finally, one steering committee member remarked that “the team of people who are employed with TRiP are dynamic leaders, problem-solvers, and collaborators—whose team approach is truly at work for these families.”

Benefits to Agencies The second question asked of steering committee members focused on the benefits of TRiP to their own respective agencies. According to some respondents, TRiP has enabled the partner organizations to advance their thinking about what creates safe communities. It has provided an avenue for operational practices to evolve and for their potential as human service providers to grow. Others felt that TRiP has provided the much-needed connection point for human service providers in Regina to start working together to better meet the needs of children and families. The integrated support coordinated by TRiP, as some described, has provided families with an “incredible wrap-around support that acknowledges different family dynamics, barriers, and interests in the healing process.”

Success Turning to a discussion on success, steering committee members felt that TRiP provided vulnerable children and youth with the opportunity to have better access to pro-social activities, overcome many barriers to services and support, and feel that they can work toward incentives that they earn all on their own. The survey responses also revealed that a number of children were provided with opportunities through TRiP that they would never have benefited from otherwise. Several steering committee members highlighted that the coordinated support process helped to stabilize a number of children and youth who have been difficult to engage in the past. Others pointed out that TRiP has also been successful in re-connecting and strengthening the relationship between children and their schools. Finally, one respondent observed that, “TRiP is successful, merely for the fact that it engages parents as part of the team—which many have fully embraced.”

Challenges

During the survey process, steering committee members were asked to identify any challenges that they felt may have had an impact on the implementation and/or outcomes of TRiP. One of the more common challenges mentioned was the lack of sustainable funding that was available for TRiP. Throughout its lifespan, the TRiP partners have been continuously seeking funds to maintain their operations. While several partners have been able to contribute to TRiP with both in-kind and cash investments, no reliable, fixed funding has been made available. Commenting on this, one respondent shared that “several government ministries are promoting upstream prevention work that identifies early signs of risk and vulnerability, yet we continue to struggle to establish some permanent, ongoing funding for TRiP programs and staff”. Another steering committee member observed that, “TRiP is at its capacity for clients and the need for more of this integrated support in our community is very clear—yet the lack of sustainable funding creates a risk in the long-term viability of this collective effort.” Another challenge raised during the survey process was that because of the long wait-list to get into TRiP, there are certain thresholds of vulnerability that are considered when accepting a child into the program. This has posed problems to schools, who after struggling to get the parent’s consent and the


child’s interest, learn that the child they had referred was either not accepted or moved off the ‘active list’. Similarly, some agencies have found it a challenge to refer a family to TRiP, only to find out they were put on the waiting list. A third challenge raised in the surveys to steering committee members concerned continuity of care for clinical and/or case-based agencies. Once a client is connected to the sector representative at TRiP, they form a client-service provider relationship. When a client has stabilized to the point where TRiP no longer needs to coordinate supports, most of the TRiP partner agencies are able to back away. However, some of the clinical and case-based agencies cannot simply transfer that client to another worker in their home agency. As a result, there is an accumulation of client files to that TRiP representative—so much that it takes away from his/her ability to contribute to the efforts for new TRiP clients. The final challenge identified in the survey to steering committee members concerned the tug-o-war between TRiP responsibilities and home agency expectations. As on respondent described, “some of the mandated agencies struggle with how to implement the overall vision of TRiP, yet also balance services at the home agency.” The reality is, staff are torn between fulfilling their obligations at the home office and staying committed to the TRiP integrated service model.

Suggestions for Improvement Following the question on challenges, the survey then asked respondents to share any ideas they had for improving 11UI and twelve&up implementation, structure, and overall implementation of TRiP process. One suggestion was for the partner agencies to consider who else in Regina could join the TRiP team, either as a sector representative or a support affiliate. Some examples given include supports in the areas of culture, tradition, immigration, addiction, sports, and recreation. Another respondent suggested the consideration of representation from Regina’s impressive non-government sector. A second suggestion offered by steering committee members was to explore different alternatives to the division of labour among the sector representatives and the case manager. At times, the latter is often overwhelmed with coordinating new intakes among the different case conference groups. More so, there is occasional confusion among peripheral agencies (e.g. C4 participants) as to who the ‘lead’ of a TRiP case is. As such, perhaps TRiP could explore diversifying the roles of team members, and consider balancing the role of TRiP staff across the lifespan of a particular case (e.g. referral, assessment, coordination, closure). A third suggestion was to find opportunities to expand the reach of TRiP, without lengthening the wait-list or over-burdening TRiP staff. One suggestion offered was for TRiP to explore ways in which some of its more committed C4 participants could absorb some of the coordination and logistics work—not as an extra task, but as part of their day-to-day work with their shared client. A related suggestion was for TRiP to be more flexible in decision-making regarding the vision of the model. According to one respondent, “this may expand the ability for each agency to support the overall partnership.”

Impact on Clients When it comes to the impact of TRiP on the clients served, the survey asked steering committee members to share what they had observed to be an impact—with understanding that the evaluation would investigate all impacts further. One observable impact shared was that TRiP provides a team of human service providers that are all working in the same direction, to help a child/youth and their


family. Having one plan, and one consistent message of support, has allowed clients to access and engage in services more consistently. Another observable impact, as reported by one steering committee member, was a reduction in risky and troubled behavior at school. Similarly, a separate respondent shared that TRiP has helped to build healthy relationships between at-risk children and adults, which has reduced school misbehaviour and absenteeism. A fourth observable impact was that by receiving strong supports and encouragement, some clients have acquired the ability and commitment to make better choices for themselves.

Impact on Collaboration Separate from the impact of TRiP on clients, the steering committee member survey also inquired about the impact of TRiP on collaboration. Overall, respondents reported a significant increase in both the appreciation of and participation in collaboration since becoming involved in TRiP. Respondent answers reveal that collaboration has helped the partner agencies experience the utility of integrated case planning, and at the same time, has improved the quality of support for clients that are mutually served. Another observation shared was that the collaboration fostered by TRiP has strengthened inter-agency relationships and understanding of one another’s strengths and weaknesses. Finally, to highlight the far-reaching benefits of collaboration to TRiP partner agencies, the following perspective of an education sector steering committee member is shared:

As a school division, we sometimes feel that we are at the centre of it all. This bares many expectations on us that fall outside the realm of education. [However], having TRiP there to link the agencies together in order to work with families and share cohesive expectations and needs for their family, is much better than working toward different goals or having too many agencies involved where we do not know how everyone is supporting the child. This has been both very positive and powerful, and is due in great part to the members of the TRiP team. They have a great ability to meet, follow process, and collaborate on the needs of families.

Next Steps

The final question asked of steering committee members invited respondents to suggest next steps moving forward. Some of the first steps mentioned by steering committee members concerned the evaluation. One respondent shared that, “I appreciate the evaluation process. I think that is has the potential to lend further value to the partnership by affirming its evidence-based approach.” Another respondent shared that, “with an objective, and assumingly positive evaluation, next steps should include a communication plan highlighting the evaluation results, projections on how greater community impacts could be achieved by advancing capacity of the partnership, and a plan to make any service delivery improvements if they are needed.” A second suggestion for ‘next steps’ included the continued pursuit of permanent funding that would allow TRiP to not only continue its good work, but return to a higher caseload and reduce waiting lists. Additional feedback around this suggestion indicated that sustained funding would expand the reach and durability of TRiP—not to mention expand community buy-in and support. The final suggestion for next steps was that TRiP start to move into the third phase of the initiative, where it adopts the Hub Model of Collaborative Risk-Driven Intervention. Establishing a Hub Table under TRiP would help to provide a rapid triage alternative for families that simply need immediate connection to services—as opposed to the longer-term integration of supports facilitated by TRiP. In reverse, a Hub


Table would give TRiP another source of clients who could benefit from the type of multi-sector coordinated support offered through the 11UI and twelve&up initiatives. Finally, the Hub Model was suggested as a next step for TRiP because it would expand the detection and mitigation of risk to other individuals or families that are vulnerable in Regina.

7.2.5 Coordinated Case Conference Participant Feedback The survey administered to 10 C4 participants had some questions that were similar to those asked of steering committee members. In addition, however, C4 participants were invited to provide feedback on questions pertaining to TRiP benefits to clients, participation at C4 meetings, barrier reduction, services mobilized, service integration, and client impact. Results from the C4 participant survey process are presented in the following sub-sections.

Overall Experience Similar to surveys completed by stakeholders, the C4 Participant Survey began by asking respondents to describe their overall experience with TRiP. For the most part, respondents reported to have “great”, “outstanding”, and “excellent” experiences with TRiP. Several respondents emphasized the good working relationship and rapport they developed with TRiP staff. To preserve the sentiment of their answers, the following responses are shared in their original form:

• The TRiP staff are so easy to work with and communicate family needs to.

• They care about the kids they work with, and that shows in the hard work they put in to mobilize supports around those kids.

• The staff are very caring and determined to help the children on their caseloads. Another theme that emerged in the responses on this topic was the benefit TRiP brought to respondents in their own work with clients. As one respondent shared, “TRiP has been instrumental in helping us reconnect with students we had trouble engaging.” Another felt that the experience has been a “welcomed addition to working with children.” Other feedback indicates that working with TRiP has allowed partner agencies to improve relations with children and parents; assist in the provision of supports that were often difficult to access otherwise; and overall, help contribute to improved student success.

Benefits of Collaboration on Clients The next question on the survey asked respondents about the benefits that the collaboration of TRiP had brought to clients supported through the initiative. In responding to this question, respondents provided a variety of answers. Some described that TRiP provided access to services that children and their families would not get otherwise. Similarly, TRiP also helped to connect and engage children and youth in pro-social activities that they would never ordinarily have the chance to enjoy. A third observation was that TRiP has provided a clear channel for productive dialogue between the family, school, and external human service agencies. Despite the broad range of responses offered on this topic, one of the common themes around collaboration was that the process by which TRiP organized supports was very beneficial to clients. To preserve the sentiment of their answers, the following responses are shared in their original form:


• TRiP is a key player in changing the trajectory for our most at-risk youth.

• They have delivered success plans with rewards that are worth working toward and have had a lasting impact on our students.

• They build relationships that are so tight they learn what our clients love, and build connection and lessons from that point.

• It has reduced the duplication of efforts and increased the cooperation between agencies which appears to have resulted in families feeling more supported.

• The families have a chance to be empowered by the professionals who are there solely caring for the welfare of their child.

Benefits of Collaboration on Agencies

Similar to the previous question, the survey asked respondents to describe the benefits of collaboration on the actual partner agencies. One area of benefits included the capacity TRiP gave to agencies to create stronger relations with clients, improve access to services, and gain a better understanding of family needs. Another benefit mentioned was the ability of TRiP to keep the various agencies connected and engaged with one another. As one respondent described, “TRiP has served as a bond which helps our various agencies keep moving forward with our kids”. A spinoff benefit to this is that TRiP brings many supportive hands to the table. As one respondent recalled, “because of the collaboration facilitated through TRiP, if one of our own agency staff are not available to help the [client], there is always another team member there to offer support.” A third area of benefit mentioned in the survey process concern the increased efficiency of collaboration. According to one respondent, “TRiP brings together all the agencies at one time, which allows us to focus on one meeting rather than potentially booking five to six different meetings.” Similarly, another respondent shared that “TRiP facilitates both the case management and coordination of service integration, and follows through to ensure all action items are completed.” This has helped to reduce some of the burden on human service providers, therefore giving them more time to support children and youth. A fourth area of benefit is the reduction in barriers faced by service agencies themselves. According to a few different respondents, the collaboration organized by TRiP helps to overcome barriers to information sharing, access to client information, client distrust, siloed thinking, and the general limitations of agency mandate. Another barrier that TRiP helps agencies overcome is limitations in solutions to help clients. As one respondent reflected, “working together has given us some new ideas on how we can collaborate to build supports for clients in ways we never thought of before.” A final barrier mentioned, was that TRiP has helped agencies overcome the limitations of their own sectors, and reach out to other human service professionals for help with their clients. To illustrate, one respondent shared the following:

It has allowed the classroom teacher to have an opportunity to voice their concerns for students to outside agency partners and parents more frequently. [In addition], it has given them a sense of relief that something is being done outside of school—for too often, teachers only see the hard work they do and sometimes feel as though they are the only ones supporting their students which can be emotionally taxing for them.


Success The next question asked of C4 Participants concerned the success they experienced with TRiP. Overwhelmingly, respondents were able to point out a number of different indicators for success concerning TRiP. Several pointed to student reengagement in school, improvement in school attendance, and improvement in school performance. Others highlighted an increase in pro-social behaviors, participation in community and school events, and newly developed interests for sport, art and culture. Most respondents pointed out that TRiP was successful at connecting clients to services and providing positive role-modeling that they would otherwise not have had access to. Another success mentioned by more than one respondent was improved communication with clients who were previously difficult to communicate with. Related to this, was increased buy-in from children and youth who seldom show interest in pro-social activities and opportunities. Similarly, one respondent felt that TRiP provided an effective and safe avenue for agencies to become involved with caregivers. A different respondent shared that, “[Trip staff] do an amazing job working with families who have not had positive experiences with school administration or staff. They have been an invaluable asset to keeping families engaged with the process.”

Challenges Aside from benefits and success, C4 participants were also asked to identify some of the challenges they observed while collaborating with TRiP. Some of these challenges included a lack of parental engagement; long waitlist; lack of sustainable funding; parents not following through on plans; lengthy process of assessments; arranging schedules among multiple service providers; and the growing caseload of TRiP staff. Another challenge mentioned by C4 participants was that some children/youth are not accepted into the program, which creates a challenge when there is no other service or supports to help. A different concern raised was that the waiting list itself was becoming a problem. As one respondent recounted, “The waiting list has also become a bit of a concern, as it has gotten quite lengthy. However, I think that speaks more to the overall need in the community and lack of funding available, rather than a lack of effort on the part of TRiP.” The final challenge explained during the survey process concerned the lack of continuity in practice among the different TRiP partners. Since each service sector has its own way of engaging clients, there emerged discrepancies between the ways in which some TRiP partners engaged clients versus others. In addition to this, some sectors (e.g., policing, family services) also face limitations in client willingness to engage. This may have resulted in cases being closed by some TRiP staff (because of lack of engagement) but left open by other TRiP staff (because they want to provide one more chance).

Suggestions for Improvement When asked to provide suggestions for improvement, respondents to the survey process highlighted a few different possibilities. The first, recommended by most C4 participants, was for TRiP to seek sustainable funding to maintain operation of the initiative. The second common suggestion was for TRiP to expand the delivery of services and supports to more children and youth in Regina. More specific suggestions for improvement include a devoted individual and family counsellor within the TRiP staff team; a full-time Catholic School Division representative (as the current representative is only with TRiP part-time); structured leisure activities that effectively engage older youth on a more regular basis; increased access to one-on-one service providers; and opportunities for TRiP families to spend time


modelling healthy pro-social families. The final unique suggestion made was for TRiP to expand sufficient caseloads to justify and maintain additional sector representatives. C4 Participation One of the key functions of this evaluation process is to determine the extent to which the TRiP model is being implemented as originally planned. Part of completing that function it to see how well the partner agencies are fulfilling their roles in the initiative. As such, the next question on the survey asked respondents to comment on how well they felt other agencies and professionals participate in the Custom Coordinated Case Conference (C4). For the most part, respondents felt that their fellow C4 participants were well-engaged and made solid contributions to the process. Respondents explained that C4 participants are creative, committed, open to suggestions, respectful, and quite appreciative of the process. According to a few respondents, the C4 meetings themselves have nurtured trust both among C4 participants and with TRiP staff. This trust, as one respondent shared, “extends beyond the individual youth case involved and creates a productive environment in other situations.” The only limitation of the C4 participation is that due to mutual ownership over the case, no real consensus emerges to finalize exactly what supports should be offered to the client. Beyond this, most respondents felt that C4 participation was quite strong.

Barriers Overcome Another important purpose of this evaluation is to develop an understanding of the barriers to service that TRiP has helped children, youth, and families overcome. In responding to a survey question on this topic, C4 participants identified lack of trust, parenting skills, activity registration fees, transportation, and systemic barriers produced by the human service delivery system. Other barriers that C4 participants reported to have been reduced by TRiP include: unsafe home environments, denial of anti-social behavior, lack of interest, misunderstanding of service mandates, and a lack of knowledge on opportunities and available supports in the community. Describing the way in which TRiP helps parents and children overcome barriers to services, one survey respondent emphasized the importance of accounting for all variables in barrier reduction, including stigma:

For many parents that have children who are struggling, trying to seek help is overwhelming and frustrating. They don’t know where to turn. TRiP helps all the agencies wrap around the family, from helping them get to appointments, to knowing what to ask when they get there. TRiP gives the students opportunities they wouldn’t otherwise have in regards to after school and summer programming, helping these students grow as individuals. TRiP removes the stigma many parents feel when they have a child that struggles.

Service Mobilization

The next question in the survey to C4 participants asked for feedback on the extent to which TRiP has mobilized services around individuals and families. Generally speaking, most respondents felt that TRiP was successful in mobilizing a majority of the necessary services around clients and their families. Some of these services include mental health, justice supports, social work, counselling, financial help, and school support. Most of the services mobilized around clients, tended to be services that the lead agency could not provide on their own.


One of the factors that enabled much of this service mobilization was that TRiP staff maintained momentum of the case plan. As described in respondent feedback, too often, case plans fall apart without consistent support and follow-up. With TRiP’s dedicated team, it was easier to mobilize and maintain services for clients. Another catalyst of service mobilization was TRiP’s ability to get buy-in with the child/youth and their family. A third factor in strong service mobilization was that because of the collaborative nature of multiple professionals working together, there was always a certain level of accountability and group pressure to stay involved and secure service connections for clients. A major barrier to service mobilization, according to C4 participants, was client acceptance and engagement of services. Another caveat in service mobilization was that some TRiP clients were already connected to multiple services. Unfortunately, adding more services to their support plan only made things more complex, rather than more effective. A final challenge with service mobilization was that in some cases, TRiP did such a great job at getting support for clients that it may have created a situation of over-dependence. In other words, TRiP made care planning and service access so easy for individuals and their families that a few failed to make the effort to lead healing for themselves.

Service Integration Separate from service mobilization, the survey also asked C4 participants to comment on the extent to which they felt TRiP has integrated services with individuals. In providing their feedback, several respondents made reference to the ongoing case planning and shared ownership over case files that has been facilitated through the TRiP initiative. As one C4 participant described, “the [TRiP] process has allowed for a more collaborative approach to service delivery with regular case conferences, clear direction, and solution-focused problem solving.” Another respondent explained that, “TRiP has facilitated case conference meetings with families and agencies to ensure that action plans are being carried out.” A third respondent noted that “TRiP has case managed, and brought teams of people working with some of our most difficult families together on the same page to make a greater, more focused impact.” According to some respondents, evidence of these integrated supports is clear when examining client improvements. According to one education professional, “TRiP has been an integral part of the success our students experience. There is nothing more powerful for a family and a student than having the whole team involved with that student around the table discussing what the successes and challenges are.” Another respondent shared that, “by offering structured leisure activities in combination with other human service supports, it became easier for youth and children to stay engaged.”

Client Impact The final question asked of respondents invited feedback on the impact that they had observed TRiP having on clients. In answering this question, several respondents explained that TRiP has helped clients overcome service barriers, become involved in programming, and feel supported by a multi-sector team of professionals. Other impacts mentioned include an increase in school attendance, boost in positive attitude, enhanced family connectedness, and improved parent engagement in school and community activities. Some individual examples of respondent feedback to this question highlight various types of impact, including: “TRiP is giving some of our most at-risk youth a shot at success”; “[TRiP] has been effective in building so much rapport that our student does not want to disappoint”; and “Because of TRiP, our student is going to school full-time and enjoying being part of activities such as camps and clubs.”


7.2.6 TRiP Staff Feedback

Instead of administering a survey to TRiP staff, the evaluator conducted individual face-to-face interviews with 11 members of the TRiP staff team. Interviews lasted anywhere from 30 to 60 minutes. In addition to all of the topics covered in the C4 participant survey, TRiP staff were also asked to comment on TRiP achievements, client outcomes, key ingredients to consider in replication, the reporting process, and any other topics they felt were relevant to the evaluation. The following sub-sections present results from interviews with TRiP staff.

Accomplishment The first question posed during the staff interviews asked respondents to describe what they have seen TRiP accomplish since becoming involved in the initiative. For the most part, the responses appeared in two themes. The first of these involved the development of process, while the second involved the development of relationships. With respect to process, TRiP staff explained that in the past 3 years, TRiP has established a disciplined and consistent process for risk detection, needs identification, information sharing, and integrated support planning. This new process has helped to deliver a quality of intake and assessment that better prepares TRiP staff and partners to understand the needs of clients and their families. This new process has also allowed TRiP to become more organized, systematic, and able to maintain momentum of the initiative. A major support to the formation of this process, according to staff, has been the redevelopment of forms, instruments, and reporting tools for TRiP. As one staff member described, “the referral forms, ICT planning sheets, and even vulnerability scoring tools, help us build greater quality of care for kids and their families.” A related support to this process accomplishment is also the TRiP Functioning Process Document. As one staff explained, “having our entire process documented not only helps our team stay consistent, but it helps both new team members and other communities learn how we engage in multi-sector coordinated support.” As mentioned, the other major area of accomplishment for TRiP has been in the area of relationships. According to staff, TRiP has formed strong working relationships between parents and service providers, and among human service professionals in Regina. Some specific examples given include relations between police and education, mental health and justice, and child and family services with all agencies represented at TRiP. The catalyst for much of this relationship growth has been TRiP’s efforts to foster service integration, share ownership over client outcomes, and most of all, get multiple sectors to work in the same building. Concerning service provider relations with clients, these relationships have been bolstered because TRiP helped bridge service gaps between families and professionals, reduced barriers to service access, and has been able to engage clients in pro-social activities in the community. Other accomplishments mentioned in the interviews with staff include the enhancement of community awareness, growth in staff unit size and infrastructure, buy-in from community partners, and interest among vulnerable families to become supported through TRiP. The final accomplishment of TRiP, as identified by staff, is that other communities have recognized the unique strengths of the TRiP model and have started to seek opportunities to learn more about it.


Benefits of Collaboration to Clients The next interview topic invited TRiP staff to discuss the benefits they observed collaboration having on their shared clients. One theme in this discussion was the support network that TRiP has mobilized around clients and their families. By bringing partner agencies together in an integrated fashion, clients gain access to a broader collective of supports. This, according to one respondent, “gives clients a true feeling of community support.” Another shared that, “our collaborative network of agencies has given clients a broad support system they can draw upon.” Related to this, another theme in the interview dialogue was that the collaboration among TRiP agencies has improved client access to supports. Several staff members pointed out that their collaborative efforts have actually helped clients gain access to services they would otherwise not have gained access to on their own. Others highlight that the collaborative process at TRiP has also empowered parents to play an active role in leading the care plan for their children. In fact, as one staff member described, “our collaboration with other service agencies has actually helped to keep parents and their children both more organized and engaged.” Finally, staff feedback also suggested that collaboration among TRiP and its partner agencies has allowed for quicker mitigation of risk, identification of client need, and faster access to services that are better informed on client need. A third area of benefit to the collaboration fostered by TRiP concerns the overall positive experience that children and parents have had with TRiP. A few staff described that traditionally, some of the hard-to-reach families did not have positive experiences with human service professionals. However, through the collaborative process, TRiP and its partner agencies have been able to foster positive experiences for clients and their families. As one staff member described, “we have given families a sense that people genuinely care.” Another respondent remarked that, “this process has empowered parents to be part of the process, which has improved their relationships with schools.” Finally, another respondent echoed that, “parents feel they have a voice in the case, which is very empowering.”

Benefits of Collaboration to Agencies Also discussed with TRiP staff was the benefits to collaboration felt by agencies involved in the initiative. Overall, four main themes emerged from the interview dialogue: improved knowledge, improved efficiency, increased effectiveness, and strengthened relationships. Regarding the first of these, staff described that the collaborative process has given them a new perspective on what each sector can bring to the partnership. As one respondent shared, “it has helped us better work with other agencies, understand each other’s limitations, and work better together”. Another felt that, “knowing what we can each bring to clients helps us make a better plan overall”. With respect to increased efficiency, staff members reported that a number of benefits of TRiP’s integrated support process has helped agencies “save time”, “reduce communication barriers”, “ease access to service”, and has “reduced the hesitance to share information and collaborate”. Interview dialogue also indicates that TRiP has allowed multiple agencies to work together, helping the same client without duplicating services. In addition, one staff member explained that, “[TRiP] keeps us on task so we can make sure collaboration continues and clients get what they need.” Another benefit to collaboration identified in the staff interview process was increased effectiveness. According to staff, collaboration brings additional tools for agencies to help clients. Some of these tools include additional information, access to resources, and improved understanding of client needs.


Collaboration also contributes to effectiveness because it brings accountability. As one respondent shared, “it is easy to let things slide when nobody knows how its going…however, when we collaborate we don’t drop the ball as much.” One benefit to this improved effectiveness is that schools feel less stressed because of the support network TRiP provides. As one respondent observed, “through collaboration, [schools] feel that they are getting something done.” The final benefit to the collaboration fostered by TRiP is strengthened relationships among partner agencies. Feedback from respondents indicates that collaboration among the agencies increases shared value among partners, improves communication, and fosters role-clarity within the community. The collaboration fostered through TRiP has helped schools and other human service agencies gain comfort in approaching one another for help. It has “added a common face for people to work with.” Finally, as one staff member shared, “the collaboration at TRiP acts like a glue that keeps everyone working together.” Benefits of Collaboration to TRiP Staff Another beneficiary to the TRiP model has been the staff of TRiP themselves. To learn how each team member has benefited from their involvement in TRiP, the interview process invited staff to provide some feedback on their experience. To preserve originality of staff comments, the following feedback is presented in its original form:

• It has allowed us to feel that we are all on the same team, which gives us a unified identity and better motivation to help families.

• It has broadened my understanding of vulnerability among youth and how we can offer better solutions to help them.

• The TRiP identity has helped us work together much more than a normal case conference.

• We have been challenged to be more innovative in our solutions and try new things.

• TRiP has really forced us to multi-task and be more responsible for fulfillment of client care plans.

• It gave me an intimate understanding of other sectors—that doesn’t really happen anywhere else.

• It has provided group-based opportunities to serve clients that I never experienced before.

• We have come a long way from where we were to where we are now.

• I have grown incredibly as a professional. This experience has changed my perspective and challenged my competence on working with children and youth.

• I have gained a better understanding of what others do.

• I have been able to see the outcomes of the children and the benefits of our work.

• It has enhanced my knowledge of what is available in the community.

• It has given me a lot of insight into other roles.

Successes When asked to describe some of the successes observed during their time at TRiP, some staff focused on the good working relationships formed between human service providers. These relationships provided an opportunity for TRiP partners to grow as a team and act effectively act in very short windows of opportunity. As one respondent described, “the ability of TRiP to build a team and surround a family is incredible.”


In discussing success, other staff highlighted the initiatives’ effectiveness at engaging children and youth. Several staff pointed to marked improvements in client relationships with teachers and other children/youth, as well as overall improvements in behavior changes. Much of this success, according to staff, is attributable to their efforts to connect clients to activities in the community as well as the School Engagement Officer position. A third area of success highlighted during the staff interview process was the improved engagement of parents in family, community, and school life. Staff explain this success as a function of their efforts to “meet parents where they are at”, “help them become part of the plan”, and “support them in overcoming barriers to service access.” Additional dialogue with staff revealed that “parents who engage in services show incredible gratitude”, and “as we increased trust with parents, we saw a reduction in overall risk.” Challenges The next question on the survey asked respondents to identify any challenges that they have become aware of regarding any aspects of TRiP. One of the first areas of challenge concerned the difficulty to attract sustainable funding for the overall initiative. Operating TRiP certainly requires in-kind support from the partner agencies. However, it also needs direct funding to stay operational. A current funding-related threat to TRiP is that the staff are already stretched fairly thin. Additional demands for multi-sector coordinated support may threaten the stability of the initiative. Another area of challenges concerned limitations with the partner agencies and their involvement in TRiP. One particular challenge is the fact that not all partner agencies have equal representation and commitment to TRiP. As such, some sector representatives are full-time whereas others are only partially available to work on TRiP activities. Another drawback is that some of the partner agencies are limited to only supporting clients that their agency is already involved with—despite the composite needs of that child/youth. A third challenging area concerns partner agency role. One of the challenges in this area is that mental health is the only sector that actually offers service provision. The police and education representatives play liaison roles, while social services and justice are mandated case managers. The consequence to this is that when files close at TRiP, all the other partners are removed from a case. However, quite often following closure, mental health continues to work on that file—making the sector representative increasingly busy. Another challenge with respect to agency role is that mandated sectors like police, social services, and justice tend to elevate stress/tension because of the client’s apprehension towards forced compliance. On more than one occasion, TRiP staff felt that they had to work extra hard to minimize the concerns of parents when these mandated agencies became involved. One way to alleviate this elevated tension could be to involve case aides to serve as liaisons for the mandated agencies (as opposed to actual social workers or probation officers). One of the most common challenge areas discussed by TRiP staff regarded process. In particular, some staff respondents explained that the front-end process of reaching out to parents and getting their buy-in is not always easy nor timely. Following initial buy-in, the intake process appeared for some, to take up additional TRiP staff time that was likely not necessary. On occasion, some staff also felt that the


assessment and start-up process took up a lot of time for those sector representatives not directly engaged in the file being discussed. A fifth challenge area discussed by TRiP staff was the growing waitlist that potential clients are put on before they begin with TRiP. Generally speaking, TRiP staff felt that managing the waitlist was a significant challenge. However, they realize that they are already working with a large number of files as it is, and do not want to undermine quality support by taking on too many clients. As one respondent reflected, “when people are referred to TRiP, and we put them on a waitlist, we pretty much just lost our opportunity to help them.” Another respondent felt that even if families do stick around after being put on the waitlist, if they sit on that list for too long (e.g., 2-3 months), they will lose interest and not engage. In addition to the above-mentioned challenge areas, TRiP staff were able to highlight several other individual challenges. Those challenges are summarized below:

• Once we become involved in a family, some parents and even teachers back off in trying to support school engagement.

• The problem is when C4 is engaged, clients stabilize. But when TRiP closes a file, the collaboration stops and clients start to slip again. It is critical to maintain collaboration.

• It has been hard to get community partners to understand what a good referral is.

• Clients have been overwhelmed at times when there are so many professionals involved.

• It seems that our home agency clients get less attention that our TRiP clients.

• Some of our agencies already do case conferencing for their clients through their home agency. It becomes confusing whether a client should be a candidate for internal case conferencing at the home agency or case conferencing through TRiP.

• It is difficult when some clients get accepted and others do not, even though their backgrounds and problems are very similar.

• Due to resource limitations, TRiP has decided not to chase every client we have initial contact with. The problem with this is that the hard-to-reach clients stay hidden and never do join the program.

• The title of case manager is inaccurate as we all play a role in leading cases.

• Considering the work of school liaisons, their title is not accurate for what work they do.

• One of the challenges for children and parents is that, at times, there are numerous professionals sitting around talking about the client’s parenting and their home affairs. This becomes uncomfortable for some clients.

• Within the community, there is still not a solid understanding of what TRiP is.

• TRiP has been forced to survive in the shadow of the Hub Model. This has made it difficult to gain support and involvement from partner agencies and funders.

Opportunities for Improvement

Following a discussion on the challenges facing TRiP, staff were asked to identify any suggestions they may have for improvement. The responses provided by TRiP staff can be grouped into several different themes. These include: structure, process, partner commitments, and additional resources. With respect to structure, some staff felt that the entire case management duties should be shared among all sector representatives who serve as case leads. This would free up more time for the case manager to support collaboration and assist in multi-sector coordinated support. Another structural suggestion was for the


steering committee to transition from being an approval and monitoring board to an active advisory board that engages in trouble-shooting, expansion strategies, and systemic barrier reduction. Concerning process, the first suggestion provided was that TRiP needs to spend more time with fewer clients to increase overall support. In doing so, at least three respondents felt that it was important for staff to find a way to get clients access to services quicker, and with less wait-times. Informing this issue further, one respondent shared that, “There is a misperception that TRiP gets kids increased preference to service access—which is not the case, these kids wait like anyone else.” Another process suggestion was that the initial assessment presentation during the Intersectoral Collaboration Team meeting is too long and detailed. Instead of going through as much detail as is summarized currently, some staff felt that a condensed version would be more appropriate—especially considering some ICT discussions do not pertain to all TRiP staff. A secondary benefit to more condensed initial assessment presentations is that more files could be introduced at that time. Another suggestion was that it become an expected practice for all of the relevant partner agencies to be at the very first Custom Coordinated Case Conference. This would make sure the family can see all of the available resources, while at the same time, putting everyone on the same page where the family’s needs and interests are concerned. One process suggestion offered by several staff was that sector representatives should not be expected to carry a client caseload from their home agency. When they are already assigned to be the TRiP case lead, it is an unfair burden to also assume that they will continue to be the main service provider for that (or any other) client. As one staff member described, “when we are managing files outside of our regular C4 duties at TRiP, it takes away from the time and contributions we can make collaboratively through TRiP.” Another process suggestion concerned the different needs and interests of older TRiP clients. As one staff member explained, “Older kids require more relationship building, which takes more time. Older kids actually want a relationship—not just to go bowling.” Another staff member highlighted that many of the clients in the twelve&up program are chronically high risk and have been in the system already. As such, more innovative and longer-term solutions are necessary to engage and support this cohort. Concerning partner commitments, several of the staff suggested that all partner agencies should make the same full-time commitment to TRiP as other partners. According to respondents, everyone involved in TRiP is equally as important. When part-time partners are not available, their absence is felt by the rest of the team. Staff suggested that not only would TRiP benefit from full-time commitments from the partner agencies, but sector representatives could contribute so much more if they were not also given case files from their home agency to manage outside of TRiP. One observation, in particular, was that perhaps sector liaison positions from mandated agencies would be a more effective case lead than the current actual mandated positions (e.g. child protection worker, probation officer). This may open up more contributions for prevention while still allowing mandated involvement to occur where necessary. The final group of suggestions for improving TRiP involved additional resources that would improve the overall reach, appropriateness, and effectiveness of the initiative. One of the more common suggestions in this area was providing mentors for children/youth. As one staff member described, “our clients are in desperate need for positive mentoring and there is really no accessible solution to meet this need in Regina.” Another suggested resource improvement was the addition of Aboriginal Elders. Having Elders, as one staff member explained, “would make TRiP a truly balanced and holistic support for vulnerable


children, youth, and families.” The final suggestions for improvement include making an online referral form accessible to families, and a translation of the TRiP website and promotional material into Cree. C4 Participation To contribute to the process aspects of this evaluation, TRiP staff were asked to describe the extent to which they felt C4 participants were fulfilling their role in Custom Coordinated Case Conferences. For the most part, staff respondents reported that C4 participants are “buying into the process and participating regularly.” According to interview feedback, many of the participants come well prepared, take their role seriously, and have shown ownership over a casefile. One respondent expressed that, “C4 participants are an important part of the process. They are on board and accountable.” Another believed that “the partner agencies have made a big commitment to attending C4, and are working together to impact clients.” In contrast to these observations, TRiP staff also highlighted a few struggles with C4 participants. One respondent explained that, “During the C4, it would be so much more helpful if the referring agent was present to give better background.” Another respondent pointed out that, “Participants of C4 don’t always have time to attend, so the TRiP representative for that agency ends up having to go—which is burdensome because we already have our own caseload.” A third observation was that despite all good intentions of both C4 participants and TRiP staff, it is very difficult to work around everyone’s busy schedule. Finally, one staff member admitted that, “participating in C4 is a challenge for low capacity organizations that only have so much time to spend on a client’s file.”

Barriers Overcome One of the main intents of TRiP is to help vulnerable children and families overcome barriers to services, supports, and activities. Interview dialogue reveals that TRiP has helped families overcome barriers related to trust for police, social workers, mental health therapists, and other human service providers. According to respondents, the initiative has also helped families confront stigma and overcome their reluctance to receiving supports. Some of the personal or situational barriers overcome include transportation, financial capacity, childcare, time management, language, comfort asking for help, and anxiety towards the human service system. More systemic barriers overcome include wait-times, navigating the system, accessing proper information, getting additional help, communicating with service providers, entrance thresholds, and registration fees. Service Mobilization Another question asked during the interview process was whether staff felt they had mobilized services around clients and their families. Feedback from respondents indicates that significant effort has gone into mobilizing proper and adequate supports for families. As one respondent described, “We have brought entire teams together, allowing parents and kids to feel supported and excited about what we can offer to them as a team.” Another recalled that “[TRiP] has given ease of access to services and has also helped provide clear direction to families to help get certain things done to help their family.” Additional feedback from TRiP staff indicate that in partnership with C4 participants, they were able to find pro-social activities for children/youth, connect families to services, and mobilize additional services along the way as needs arose. In reflecting on a limitation in service mobilization, one respondent shared that, “despite our efforts to mobilize services, we cannot force families to accept them. This requires a voluntary effort on their part.”


Service Integration

Another topic discussed with respondents was the extent to which they felt TRiP had integrated services in a way to help clients and their families. Overall, TRiP staff felt that their collaboration with C4 participants fostered the type of service integration that allowed for a better understanding of client needs and a quicker mitigation of risk. By working together, and sharing ownership of the client file, TRiP staff felt that “they allowed clients to see the genuine utility of human service providers—which may be different than past experiences.” One benefit of this service integration, according to several staff members, is that there is tremendous accountability among the human service agencies, and with the client. As one respondent shared, “when we integrate, the partners do a better job staying on top of things and the parents actually make an effort to help build solutions for their family”. Another benefit of service integration is that “clients are getting the multi-sector support they need for their very diverse and complicated issues.” During the interview process, it became clear that a major enabler of the service integration coordinated by TRiP, is the fact that TRiP staff are all located in a shared space. This has allowed for increased information sharing, collaborative solution building, and an increased ability to monitor client needs and service access. Another enabler of service integration, as one respondent shared, “is the willingness and commitment of C4 participants to mobilize around an individual and provide custom, coordinated solutions to reduce their overall vulnerability.”

Client Outcomes

One of the topics TRiP staff were most comfortable speaking to was the outcomes they observed for TRiP clients. Some of the outcomes mentioned by respondents include behavioural improvements, commitment to school, increased school attendance, and an elevation in student confidence. Some staff report that their clients have increased their reading, have become more interested in school, and have even made improvements in their overall school performance. Concerning behaviour, staff report TRiP clients to have shown improved self-esteem, reduced anti-social behaviour, increased community engagement, and an overall improvement in the way clients treat school staff and other children. In fact, several respondents said they heard from parents and schools about behaviour improvements in children and youth. In addition to outcomes on children and youth, TRiP staff also shared their observations on parent outcomes. Some respondents highlighted increased parent involvement, parent ownership, and parent confidence and trust in working with human service providers. One staff member shared that “parents are feeling better about themselves and the progress their children are making.” Another explained that “parents have started to take a lead role in the process of building a plan for their family.” Other outcomes mentioned during this discussion include the development of protective factors, client connections to service, reduced barriers, service engagement, and improved relationships with human service providers. With respect to the latter, one respondent shared that, “families look forward to C4 meetings because everyone is sitting in one room.” Another staff member explained that. “TRiP has bridged the gap between parents and agencies, which has improved relationships with our agencies, and has continued to increased family stability.”


One of the challenges with respect to observing client outcomes is that some of the clients supported through TRiP are chronic high-risk families. As such, they require ongoing case management and support beyond what is typically provided to other children. One example given was that “clients with social services involvement are usually already at a point of elevated chronic risk, so it is harder for us to observe any impacts on those individuals.” Another staff member shared that, “it is hard for us to fulfill our immediate goals with some families, only because they are affected by such deep problems that require permanent support and ongoing care.” Key Ingredients When it comes to replicating human service models, one of the main factors to explore are key ingredients that contribute to success in existing models. During the interview process, staff were asked to identify key ingredients to success of the TRiP model. Their responses have been organized into 7 themes appearing in Table 34 below:


Table 34. Key Ingredients of the TRiP Model by Theme THEME KEY INGREDIENTS

Perspective • Think outside the box and be innovative in solutions.

• Make client needs a collective priority.

• No egos can be involved.

• Never give up, success does not come easy.

• Must have a strong passion for the job and strong satisfaction with team opportunities.

• Realize that this is natural work for human service providers.

• Make a commitment to get the work done.

Resources • You must have an integrated office space to stay connected.

• Financial resources to overcome client barriers.

• Strong relationships with the human service community.

• Transportation and childcare capacity.

• Funded positions to coordinate process, manage data, and mobilize partner agencies.

• Staff representation from every sector.

• Mentors and positive role models.

• An experienced external evaluator.

• Strong and supportive steering committee.

Personnel • Strong human service providers.

• Staff who are really engaged and good at client advocacy.

• People that are very organized, with A-type personalities who get things done.

• Committed positions to run the initiative, coordinate services, and carry out activities.

• People who are flexible, a team player, organized, hardworking, and innovative.

• Sector representatives must have capacity to contribute and commit.

• Must have the willingness to collaborate.

• Good people who can work together, challenge one another, and come up with best plan.

Team Work • Good collaboration and teamwork.

• Get back to one other in a timely fashion.

• Buy-in to a collaborative environment.

• Commitment to working with one another in a common space.

• Balanced representation of different sectors.

• The team must understand that this can be accomplished.

Sector Partners

• Must have all the right agencies involved.

• Must have a strong relationship with the school boards.

• Find the right person in each agency who is committed and willing to work collaboratively.

• Have the entire agency committed and working together.

• Need investment from all sectors.

Preparation • Must look at what other collaborative initiatives have done.

• Be aware of potential risks and challenges.

• Develop a solid framework and action plan.

• Make sure partners and the community are informed.

• Engage community partners in the development and design process.

Process • Regular and consistent meetings.

• A clear, low threshold, disciplined and well-communicated process.

• Ongoing evaluation, data collection, and performance monitoring.

• Use evaluation to shape further development, improvement and ongoing buy-in.


Reporting Process One of the major changes to TRiP during the evaluation period was the development and implementation of a new reporting process. This process was designed to streamline information gathering, build efficiencies in case planning, better understand client needs, measure client performance, and track program outputs and outcomes. During the interview process, staff were asked to comment on the overall impact of the new reporting process on TRiP. Overall, staff felt that the new reporting process has helped TRiP to become more fluid, resourceful, and organized. As one respondent commented, “The forms helped us become streamlined, tighten up our overall process, and become more efficient between the point when we assess client needs and integrate services and supports.” Another respondent claimed that, “the reporting process improved TRiP’s operation without sacrificing the quality of information being collected and shared.” Overall, the strength of this process, according to a different respondent, is that “it is user-friendly, fairly clear, and everyone knows the process and steps of how to do it.” Other feedback on the reporting process emphasized how it has contributed to increased effectiveness of TRiP. For example, one respondent shared that the reporting process “has kept us on top of things and helped us better understand client needs.” A second respondent shared that “working with a developmental evaluator on this reporting structure has helped to get things organized, structured, conceptualized, and consistent…we have really come a long way with this initiative.” Other observations on how the new reporting process benefits TRiP include:

• The reporting has helped to instill a solid process and keep us organized.

• It helps with new people coming to understand what good practice is.

• It gives us better data from the referral sources.

• It has helped us establish a structure and way of doing business.

• The process alleviates a lot of the subjectivity and bias in our old system.

• The ICT Action Plan gives us a clear picture of what is going on—the instrument has high utility.

• There is huge value in the overall consistency of the reporting process.

• The reporting process has helped to maintain consistency and fidelity in the model.

• The data from our reporting process provide so much more information and capacity to help. Despite the overall satisfaction with the reporting process, there were some difficulties in implementing the process. One problem for some respondents was that although the TRiP forms are helpful and effective, some staff have a complete reporting process they have to manage within their home agency. This takes away from the time they spend on TRiP reporting. Another challenge is that even if TRiP staff try to manage both TRiP reporting and that of their own agency, there is very little overlap. As such, some staff end up having to complete two different reporting processes on the same client. One of the weaknesses of the reporting process, according to one respondent, was that TRiP clients and parents do not always recognize that it was TRiP that actually coordinated their services and supports. As such, it becomes difficult to get parents and children to provide accurate or relevant feedback when they may not entirely be sure what TRiP is.


Additional Feedback At the end of each interview, TRiP staff were asked to provide additional feedback that they thought would be relevant or supportive to the evaluation of TRiP. Some of the dialogue offered indicated that collaboration of services among multiple agencies is not easy to coordinate. When multiple service providers carry different schedules and client caseloads, it does become difficult to keep everyone on task and together. Considering this, some staff felt that the entire collaboration process itself was an impressive outcome—largely because integrating multiple complicated service sectors can be inherently challenging. Helping staff get through these struggles, according to one respondent, is that “[TRiP staff] stay focused on the positive and do not get dragged down by the challenges and frustrations of multi-sector work.” The final comment offered during the interview process was that “TRiP may actually be easier to implement in a small community where collaboration is a must because they have fewer services.” 7.2.7 Evaluator Observations Throughout the evaluation period, the evaluator was able to observe TRiP and its partner agencies in different parts of the TRiP process. Most observations were made during Intersectoral Collaboration Team (ICT) meetings and Collaborative Custom Case Conferences (C4s). Other observations were made during community outreach and various client support activities conducted onsite at TRiP offices. Overall, the first impression of TRiP was that the team itself had strong synergy, collaborated effectively, and was successful at building strong inter-agency relationships. Indicators of this were the regular check-ins, ongoing consultation among the partners, and the emergence of a collective identity in TRiP. Another initial impression was that the team developed shared ownership over clients and their families. Not once, in any of the observations, did a client’s file appear to belong to one particular agency. In fact, after observing several different TRiP activities, it became clear that the multiple organizations involved in TRiP generated a shared ownership over client outcomes—both successes and challenges. This may very well account for the strong teamwork, ongoing communication, and comprehensive support also observed in TRiP. It may also account for much of the positive feedback received from TRiP clients and their caregivers. A third observation of TRiP was that the team itself very much benefitted from having a shared office space. Although collaboration can certainly occur outside of shared work environments, being in the same facility seemed to foster a united approach to problem-solving, client engagement, and mobilization of C4 partners. Related to this was the observation that TRiP’s collective identity, overall capacity, and confidence to pursue innovative solutions were fostered by having a shared coordinator, strategist, and administrative support that served as a backbone for the entire team—regardless of agency affiliation. In fact, having the coordinator and strategist act as a buffer between TRiP staff and their respective steering committee representatives, combined with shared office accommodations, may have minimized staff reluctance to work outside of their sector silos—which many collaborative initiatives struggle to overcome. Another observation made during the evaluation period was that although TRiP is focused on supporting children and youth, it became very clear in observing the ICT and C4 meetings that much of the work being done to support children/youth involves supporting their caregivers. In fact, some caregivers reported in their exit survey that TRiP was instrumental in triggering changes in parenting behaviour,


styles, and approach. This became particularly evident later on, when C4 teams would continue to support caregivers even after their child had been connected to supports, services, or activities. One of the challenges observed within TRiP was that half of the staff were full-time commitments to the initiative while others had half-time or more than half-time commitments at their home agency. While no relationship-divide between these two groups was observed, it was clear that the part-time commitments were limited in the energy and contributions they could put toward TRiP. In fact, these observations were supported by half-time staff feedback indicating that they themselves felt they could do more if they were not responsible for carrying files at their home agency. As a result of these limitations on half-time staff, the full-time staff were forced to enhance their support. Although this minimized disruption in TRiP support to clients, it did take away from the truly multi-sector support that could be offered to clients had all TRiP staff been full-time. Another observation, which could be considered more of a hurdle than a challenge for TRiP, was the constant search for operational funding and renewed in-kind commitments. Although sustainability is a reality that all social initiatives must confront, TRiP spent much of the last 3 years balancing self-development, growth, and refinement of process, with trying to convince funders to invest in the work being done. Had larger and longer funding commitments been put in place from the start, TRiP staff may have been able to get to where they are today sooner, if not further ahead.


8.0 SCHOOL ENGAGEMENT OFFICER: SPECIAL ANALYSIS An integral part of The Regina Intersectoral Partnership’s (TRiP) delivery of both the 11 and Under Initiative (11UI) and Twelve and Up (twelve&up) program is the School Engagement Officer (SEO). This position, funded as a two-year pilot position, is designed to help reduce the personal, situational, and systemic barriers to school engagement, attachment, and achievement. Although part of the broader TRiP team of sector representatives who collaborate to identify and meet the needs of at-risk youth, the SEO plays a specific role in mitigating conditions, attitude, and risks that lead to school disengagement (i.e. lateness, absenteeism, disinterest, achievement). During preparation of this evaluation report, it became clear that there was value in providing a special analysis of the SEO position. As such, in June of 2017, the analytical brief that makes up this section of the evaluation was provided as a stand-alone document to TRiP9. Both the independent paper and its inclusion in this larger evaluation provide an overview of the SEO position, offer a quantitative understanding of the clientele being supported by the SEO, examine client outcomes, and summarize some of the perspectives held on this position. The intent of this special analysis is not summative in nature. In other words, it does not determine whether or not the SEO position is effective or ineffective. Rather, the intent is to provide an objective understanding of the contributions of the SEO position to the broader TRiP initiative. In that, this document will highlight some of the benefits to TRiP clients and partner agencies involved in the 11UI and twelve&up initiatives.

8.1 ABOUT THE POSITION The SEO position was designed to target school disengagement, as past research finds school truancy to be one of the leading catalysts of anti-social behaviour and criminal activity among youth10. Past evaluations of the 11UI initiative11, supported by additional research on youth vulnerability12, identified that a proactive and assertive effort to reduce the barriers to school engagement was required to improve overall success of the TRiP model. By design, this position was focused on accepted TRiP referrals in two areas. The first was children and youth with significant school absenteeism, lateness, and/or other disengagement behaviours. The second was youth being released, or who have recently been released, from custody. During the evaluation period, a majority of clients supported by the SEO fell into the former of the two categories. With respect to engagement, the SEO becomes involved in client files through one of two ways. The first is when at the Intersectoral Collaboration Team (ICT) meeting, a new referral identifies that school attendance, achievement, or engagement are concerns for the client. The second is when the TRiP team of sector specialists is already working on an open TRiP case, and during their collaborative support, recognize the need for additional supports by the SEO.

9 Nilson, C. (2017). 11 & Under Initiative and Twelve & Up: School Engagement Officer Component (Special Analysis). Prince Albert, SK:

Living Skies Centre for Social Inquiry. 10 Office of Juvenile Justice and Delinquency Prevention: http://ojjdp.ncjrs.org/truancy/index.html. 11 Wright, K. (2014). 11 & Under Initiative (11UI) Evaluation, 2013-2014: Final Draft. Regina, SK. 12 TRiP. (2014). The 11UI Conceptual Framework—Regina Intersectoral Partnership: Prevention and Reduction of Crime. Regina, SK: The

Regina Intersectoral Partnership.


Once involved, the SEO undertakes a variety of supportive actions with and on behalf of that student. Some of these actions include the following:

• well-being checks/home visits

• development and monitoring of student incentive charts

• delivery of incentives (e.g., hot lunch, special treat)

• reduction of barriers (e.g., registration fees, transportation)

• support school in addressing behavioural issues

• provide parents with reminders and encouragement

• support family through barrier reduction and risk mitigation

• monitor student progress

• participate in case conferences with other sectors/agencies

• provide pro-social activities for students (e.g. reading, sports, lunch-outings)

• provide positive role-modeling

• direct support to students in the areas of literacy, numeracy, and at-school specific events

8.2 CASE INVOLVEMENT To gain an understanding of the type of cases involving support from the SEO position, a special analysis of data from the TRiP Client Database was conducted. This analysis involved a separation of files involving the SEO from those not involving the SEO. Following this, basic descriptive frequencies for each cohort were generated. The data examined herein are from TRiP client files that were open and/or closed between September 1, 2015 and May 31, 2017. As Table 35 shows, of the 77 files involving the SEO, a slight majority were still open at the time of this analysis. Further data show most of the files involving the SEO were referred from the education sector (see Table 36). Similarly, once a TRiP file becomes open, most continue to be led by the education sector (see Table 37).

Table 35. File Status (as of May 31, 2017)

SEO Involved (N = 77) SEO Not Involved (N = 283)

File Status N % N %

Open 46 59.7 54 19.1

Closed 31 40.3 228 80.6

Wait-List 0 0.0 1 0.4


Table 36. Referral Source


Referral Source N % N %

Catholic Education 13 16.9 35 12.4

Public Education 33 42.9 100 35.3

Justice 4 5.2 18 6.3

Police 13 16.9 57 20.1

Social Services 3 3.9 7 2.5

Health 4 5.2 33 11.7

Caregiver 7 9.0 26 9.9

Outside Agency 0 0.0 7 2.5

Table 37. Lead Sector


Lead Sector N % N %

Catholic Education 10 13.0 12 4.2

Public Education 49 63.6 79 27.9

Justice 3 3.9 6 2.1

Police 0 0.0 1 0.3

Social Services 9 11.7 11 3.9

Health 6 7.8 10 3.5

No Lead 0 - 164* 58.0 * Cases that are closed, rejected, or in the referral stage do not have Leader Sectors assigned.

8.3 CLIENT OUTCOMES

While the main purpose of this special analysis is merely to demonstrate the nature and type of client cases involving the SEO, some insight into client outcomes may also be valuable. To examine client outcomes, a number of different methods were used. The first of these involved case studies of 30 non-randomly selected student files with SEO involvement (chosen alphabetically). The source of data for these case studies were the School Engagement Summary forms completed by the SEO during her provision of monitoring and support for each client. Data gathered from these forms included the original problem requiring SEO involvement (e.g. absenteeism, lateness, both, other); the number of actions (e.g. visits, meetings, advocacy, trouble-shooting, incentive delivery) performed by the SEO per client; and the current status of the client relative to their original problems. The latter of these variables was determined through a coding scheme involving the following categories of client progress towards school engagement: worsened, stayed same, improved, and undetermined13. As Table 38 demonstrates, the most common presenting problem among clients in the study group was “absenteeism” (n = 13), followed by “other” (n = 9), and to a lesser extent “lateness” (n = 4) or “both absent and late” (n = 4).

13 An inter-coder reliability test was conducted to verify accuracy of the evaluator’s coding scheme.


Another finding highlighted in Table 38 is that there was quite a range of actions within the study group. On the low end, the SEO undertook 3 actions for one client. On the high end, the SEO undertook 57 actions for one client. Much of this range has to do with the time in which a client is enrolled in the program, their receptiveness to the SEO’s support, and the involvement of other TRiP staff in a client file. For nearly half (n = 16) of the clients in the study group (n = 30), the SEO undertook between 3 and 10 actions for each client. For a slightly less number of clients (n = 14) in the same study group (n = 30), the SEO undertook between 11 and 57 actions for each client. Finally, Table 38 reports on the client progress—as interpreted by the progress notes captured in the School Engagement Summary form. Within the study group (n = 30), almost half (n = 14) of all clients appeared to have improved their overall engagement in school. Indicators of such improvement included “attending class”, “coming to school on time”, “being less disruptive in class”, “following incentive plan”, and “showing more respect.” In contrast, the progress of 5 clients was coded as having worsened. Indicators for this outcome include “quit coming to school”, “behaviour has deteriorated”, “student has become disengaged.” As Table 38 also shows, some clients showed neither progress nor a worsening in their status. Consequently, these files were coded as stayed the same. Finally, there were 5 cases marked as indeterminate because there was insufficient information within the most recent School Engagement Summary to make a fair and accurate determination of progress.

Table 38. Presenting Problems, Actions, and Progress of SEO Client Files (N = 30)


Presenting Problem

Absenteeism Lateness Both Other

13 4 4 9

43.3 13.3 13.3 30.0

SEO Actions 10 or less 11 to 20 21 or more Lowest Highest Average

16 6 8 3

57 17.1

53.3 20.0 26.7 na na na

Client Progress Improved Same Worsened Indeterminate

14 6 5 5

46.6 20.0 16.7 16.7

Another method used to track client outcomes is an examination of data gathered from the Post-TRiP School Report. This report, to be completed by school staff, gathers data on student attendance, student behaviour and student performance. As Table 39 illustrates, of the 20 clients who both received SEO support and have a completed Post-TRiP School Report in their file (as of May 31, 2017), the attendance of 18 clients either “stayed the same”, “improved”, or was considered “good”. In contrast, only 1 client had poor attendance following SEO support.


Table 39. Post-Support School Attendance

SEO Involved (N = 20)* SEO Not Involved (N = 33)*

Attendance N % N %

Poor 1 5.0 1 3.0

Same 6 30.0 10 30.3

Sporadic 1 5.0 0 0.0

Improved 6 30.0 9 27.3

Good 6 30.0 13 39.4 *N = number of client files closed and rated on school attendance.

Regarding school behaviour following TRiP support, 6 clients were observed to have “improved” their behaviour, 7 stayed the “same”, 2 were considered “good”, and only 3 were considered to have “poor behaviour” following TRiP support (see Table 40).

Table 40. Post-Support School Behaviour


Behaviour N % N %

Poor 3 14.3 1 2.3

Same 7 33.3 3 8.6

Average 3 14.3 8 22.9

Improved 6 28.6 7 20.0

Good 2 9.6 16 45.7 *N = number of client files closed and rated on school behaviour.

With respect to overall school performance among clients supported by the SEO position, school staff examined both strength in literacy and in numeracy. Results of the analysis reveal that 15 of 19 clients fall between “beginning” or are currently demonstrating strong performance in school. In contrast, 4 clients demonstrate “insufficient” performance or no change in performance (see Table 41).

Table 41. Post-Support School Performance


Performance N % N %

Insufficient 1 5.3 0 0.0

Same 3 15.8 3 9.4

Beginning 3 15.8 5 15.6

Progressing 6 31.6 9 28.1

Improved 1 5.3 3 9.4

Meeting 4 21.1 11 34.3

Good 1 5.3 1 3.1 *N = number of client files closed and rated on school performance.

An important note in examining Tables 39, 40, and 41 is that readers should not interpret the comparisons between SEO Involved and SEO Not Involved in relative terms. There is a high probability that clients presenting problems with school engagement are automatically referred to the SEO. As such, we must accept the likelihood that clients supported by the SEO (as reported in Tables 39 - 41) may have faced additional barriers to school engagement than clients not served by the SEO, and


consequently, require different intensities of support to reach comparable outcomes as clients not requiring support of the SEO14. As the TRiP team continues to work with vulnerable youth, client files become closed for a variety of reasons. These could be because the client is stabilizing and no longer needs supports. It can also mean that a client has failed to engage in services, or simply has refused services all together. Table 42 shows that of the 31 client files that have closed where the SEO was involved, slightly over half (54.8%) of these files have been closed because the student moved away or was not engaging. In contrast, at least 32.3% of client files with SEO involvement, were closed because the student had either “stabilized” or had the “potential to stabilized”.

Table 42. Reason for Case Closure

SEO Involved (N = 31)* SEO Not Involved (N = 228)

Reason for Closure N % N %

In Custody 2 6.5 15 6.6

Moved Away 6 19.4 17 7.5

Not Engaging 11 35.5 55 24.1

Refused Consent 0 0.0 2 0.9

Refused Services 0 0.0 31 13.6

Screened Out 1 3.2 15 6.6

Unable to Locate 0 0.0 13 5.7

Potential to Stabilize 7 22.6 39 17.1

Stabilized 3 9.7 9 3.9

other 1 3.2 32 14.0

8.4 FEEDBACK

To provide additional understanding of the SEO position, various sources of qualitative data have been used for this special analysis. These sources include human service providers, parents, TRiP staff, and clients. Collection of these data also involved multiple methods. For the purposes of this special analysis, data were gathered from letters and emails sent to TRiP by clients and other human service providers; completed surveys the evaluator received from human service providers; interviews the evaluator conducted with TRiP staff; and feedback from parents that was captured through TRiP’s ongoing reporting process. The first source of feedback explored herein is anecdotal letters and emails from human service providers and clients engaged by the SEO. In their efforts to gather a better understanding of the SEO component, TRiP staff reached out to human service providers and their clients to hear perspectives on the position. To share their remarks, Table 43 summarizes the comments by clients and human service professionals made about the SEO position.

14 One question worth further investigation is that if we can assume that TRiP clients supported by the SEO face additional barriers to school

engagement than do TRiP clients not supported by the SEO, is it the work of the SEO that has yielded similar outcomes in school attendance, school behaviour, and school performance once TRiP files are closed? After all, Tables 39, 40, and 41 do show similar client outcomes between SEO Involved and SEO Not Involved clients.


Table 43. Feedback TRiP Collected on SEO Position by Data Source

Data Source Summary of Feedback

Clients • Ever since I started working with [the SEO], I’ve been given a lot of opportunities for camps and sports that I wouldn’t have had before. I think it is cool.

• She helps me do things I couldn’t do because I had no money or access. It makes me feel better about myself.

• She helps me read and improve my reading strategies.

• My mom hasn’t been so stressed knowing that I have more support and someone to talk to.

Human Service Professionals

• Some of our students have viewed the police in a very negative light. This has really changed that, by having a police officer build a relationship with them—and to be someone they can count on for support and trust.

• Developing this relationship with a law enforcement officer has been critical to our student’s success. Our student values [the SEO] as a safety net, and as someone to go to during ups and downs.

• [The SEO] goes above and beyond for one of our most at-risk female youth. As a result, our student attends school every day. She feels better going to high school knowing that there is one connection in her life that will remain safe.

• Since our student has begun working with [the SEO], his attendance has improved and he has gained more confidence. During class discussion, he is more engaged and eager to participate.

• This is a great testament to the solid, professional, and leadership-inspired policing that should be done on a daily basis.

• Since the involvement of [the SEO], our student has improved her pro-social behaviours.

• Our student has been showing improved work habits in class to earn her special time with [the SEO].

• We are seeing signs of maturity and are reaffirmed of the right direction for this child after each interaction with [the SEO].

• Lately, our student has started to talk about careers and futures, which is amazing. Prior to that, she didn’t think nor care much about the future.

• This position has helped students increase their attendance and engagement in school.

The other sources of feedback on the SEO position involve three different cohorts: human service providers that participate in TRiP’s custom coordinated case conference (C4) group (n = 10), parents (n = 5), and TRiP staff (n = 8). The data gathered from human service providers was collected through an emailed survey sent by the evaluator (C4 Participant Survey). Data from parents was collected through the ongoing reporting process (Caregiver Survey). Finally, feedback from TRiP staff was gathered through face-to-face interviews conducted by the evaluator in March of 2017. Specific questions about the SEO position were not posed to any of these three cohorts. However, through a concentrated analytical process, feedback about the SEO position was pulled from the three different sets of data. Similar to the previous table, Table 44 summarizes observations of the SEO position from human service providers, parents, and TRiP staff.


Table 44. Feedback Evaluator Collected on SEO Position by Data Source

Data Source Summary of Feedback

Human Service Professionals

• Our own clients have really been supported by [the SEO], particularly in ways that we could not.

• The position has helped support parents who struggle with getting their child interested in and committed to school.

• [The SEO] helped to open up sport and recreation activities for children that their families could not afford.

• Our students have increased their time at school from as little as 40 minutes to full days.

• A strong relationship between [the SEO] and our students has really increased school engagement for those children.

• [The SEO] has actually served as a third-party bridge between families and schools. This has been helpful, especially with hard-to-reach families.

• [The SEO]’s incentives for students has not only promoted attendance and punctuality, but has given children access to recreational opportunities they wouldn’t have otherwise.

• There is value in the police playing this type of role in the community—instead of just chasing criminals.

• While I see tremendous benefit to helping families reduce the barriers to school engagement through a dedicated position, I am still not certain of why a police officer should hold this position.

Parents • Helped my son overcome challenges getting to school.

• Built a strong relationship with my kid, which he responded well to.

• It would have been nice if [the SEO] would have been involved last year when my child was missing school more.

• [The SEO] really helped my son get a chance to go to camp and do sports.

TRiP Staff • The SEO position helps build strong relations between at-risk youth and the police.

• It has helped to better engage the Regina Police Service with the community—as an agency that vulnerable youth can trust.

• The ongoing support of the SEO actually helped parents pull it together more—they felt more accountable to make an effort and get their children to school.

• [The SEO] helps TRiP itself offer more persistent support and encouragement to families who need it the most.

• [The SEO] role has provided youth a positive experience with police.

• It has moved the police from a traditional reactionary role to a truly prevention-based role in the community.

• The downside of the position is that families—and even schools for that matter—take a step back from encouraging attendance and expect our SEO to do the heavy lifting.

• We could use more structure around the SEO position, simply to make access to that support more predictable for new families joining our programs.

• Having a police officer in this role makes school attendance the interest of more than just educators—but that of the broader community safety network in Regina.

• The SEO’s involvement helps highlight to youth and their parents, the importance of school attendance and engagement.

• The SEO position provides a unique incentive-driven approach to get kids back into their desks—this is a rare but effective technique in our broader network of social and community services.


8.5 SCHOOL ENGAGEMENT OFFICER SPECIAL ANALYSIS RESULTS SUMMARY As mentioned in the opening of this section, the purpose of this special analysis is to shed light on the impact and contributions of the School Engagement Officer position within the overall TRiP initiative. The findings presented herein are not summative, nor are they meant to be conclusive in any manner. What they do provide, however, is a qualitative and quantitative understanding of the work of the SEO, the observed impact of this position on clients of TRiP, and the benefits of this position from the perspective of human service professionals, parents, and TRiP staff. Through this analysis, we have learned that the SEO position is intended to build rapport with high-risk youth who are vulnerable to school disengagement. Once rapport is established, the SEO works with youth and their families to reduce barriers to school engagement, improve access to pro-social activities, establish and offer incentives, and monitor student progress. During the referral/intake process, a majority of clients that the SEO becomes involved with are referred from the education and policing sectors. Moving forward, the education sector continues to take the lead, with the SEO playing a key role in the ongoing coordinated support of that client. At the time of this special analysis, the SEO was involved in 46% (n = 46) of the 100 files currently open with TRiP. Of all closed files (n = 259), the SEO was involved in 11.9% (n = 31) of these15. The leading problem to address in the files involving the SEO was school absenteeism. To mitigate this issue, the SEO undertook a variety of actions, including family visits, meetings with human service professionals, mentoring, reading breaks, client advocacy, sports, recreation, troubleshooting, progress monitoring, and incentive delivery. On average, the SEO performed 17 actions with each client, with a range from 3 actions on the low end to 57 actions on the high end. With respect to client outcomes, a case study analysis of 30 student files involving SEO support revealed that 46.6% of clients improved their overall progress towards school engagement, while 20% stayed the same. Quantitative results from internal program reporting completed by school staff report that attendance has improved or is considered good among 60% of youth supported by the SEO. Similarly, at least 11 of the 21 SEO clients with completed files show in-school behaviour to be average, improved, or good. Finally, school staff report that of the 19 SEO clients with completed files, 15 (79%) of them exhibit school performance (numeracy and literacy skills) that is considered beginning, progressing, improved, meeting, or good. Additional data on closed client files involving SEO support show that 32.3% of clients end up stabilized or establish the potential to become stabilized. Relative to those files without SEO involvement, only 21% of clients without SEO support end up stabilized or demonstrate the potential to stabilize. Of all closed files involving support from the SEO (n = 27), 66.7% showed an improvement in the client’s overall vulnerability score, indicating an increase in community engagement, increase in school engagement, and/or reduction in risk factors. Turning to qualitative data examined in this special analysis, one source of data explored in this brief report was a collection of emails and letters that TRiP staff gathered from human service professionals and clients. Feedback within that dialogue indicated that the SEO position helped improve client access

15 One possible explanation for why the SEO has only been involved in 11.9% of closed files is because the SEO tends to be involved with harder-to-engage clients, who also happen to require longer coordinated care periods to stabilize. Another explanation is that the total closed files

(n = 259) also includes “rejected cases” that the SEO would never have had a chance to engage.


to recreation, sports, and community events. The dialogue also revealed that the SEO position has helped to build strong relationships between at-risk youth and the Regina Police Service. Some responders to TRiP’s request for information revealed that the SEO provides support to youth in ways that they are traditionally not supported. According to observers, this has resulted in improved attendance, school behaviour, and overall school/community engagement. Finally, qualitative data gathered by the evaluator provides additional understanding of the contributions that the SEO position brings to TRiP. Feedback from human service providers suggests that the SEO position serves as an extra resource that schools can access from outside of their own sector. The SEO was described as being effective at engaging clients and getting them involved in pro-social activities within the community. Additional feedback from human service professionals also points to the strong relations that the SEO is building between the Regina Police Service, at-risk youth, and the broader community. Parents of TRiP clients explained that the SEO has helped their child overcome barriers to education and pro-social activities. According to some parents, the support of the SEO has also helped children/youth build confidence, make better personal decisions, and genuinely want to improve behaviour and performance. Interviews with TRiP staff revealed the SEO to be a valuable asset to the TRiP team. In particular, the position was described as one that provides concentrated support to high-need children/youth in the community. Other feedback indicates that the SEO position allows the police to play a genuine role in prevention, and as part of that process, build strong relationships with a highly vulnerable cohort. Requiring additional attention is more clarity on why the position is filled by a police officer, and what options are available to better organize and structure the SEO service delivery model to ease SEO access for new clients to TRiP. Overall, the findings of this special analysis suggest that there are both measurable and observable impacts and contributions of the SEO position which are positive. Quantitative data from multiple reporting sources, and qualitative data collected from four different cohorts, suggest that the SEO position brings great value and strength to the TRiP model. Further summative analysis may yield more accurate understandings of the SEO’s overall effectiveness. For the purposes of this evaluation brief, however, there is sufficient evidence for continued support of the SEO component as part of the broader TRiP approach to delivering the 11UI and twelve&up initiatives in Regina.


9.0 SUMMARY OF RESULTS The results of this evaluation provide considerable details on the client group, service delivery, outcomes, and challenges experienced in implementing TRiP. The following sub-sections summarize the main results from two parts of the methodology: internal reporting process and primary data collection. 9.1 INTERNAL REPORTING PROCESS RESULTS SUMMARY Analysis of data captured through TRiP’s internal reporting process revealed information on clients engaged, services delivered, and outcomes generated. Data on the target group shows that of the 360 clients referred to TRiP, 79% were male and between the ages of 5 to 14. The most common risk factors affecting TRiP clients are anti-social behaviour, criminal involvement, mental health, physical violence, and parenting concerns. Some of the major service needs identified through the analysis include recreation, mental health, counselling, mentorship, and parenting support. Some of the main service and activity barriers affecting TRiP include financial barriers (e.g. registration fees, equipment costs) and personal barriers (e.g. distrust, attitude towards help, cognitive disability). Finally, in examining Needs-Based Assessment Scores of TRiP clients, 70% of TRiP clients showed high composite needs requiring C4 support. Once invited to participate in a TRiP program, clients were assigned a case lead who coordinated ongoing multi-sector support. Most often, the education sector led coordination of collaborative efforts, with mental health and social work professionals assisting. In mobilizing other human service providers to support shared clients, TRiP staff engaged over 40 different types of professionals in Custom Coordinated Case Conferences. During the TRiP support process, 33 different clients received support engaging in 98 different types of pro-social activities. Throughout the evaluation period, 59% (n = 163) of engaged clients were connected to services, while 77% (n = 109) were provided with support in overcoming barriers to service. In addition, 49% (n = 123) of engaged clients were able to participate in activities while 75% (n = 107) were provided with support in overcoming barriers to activities. With respect to client outcomes, analysis of internal reporting data show that among closed files of clients who engaged in TRiP, 94% (n = 121) had ‘maintained’, ‘improved’, or achieved ‘good’ school attendance following support from TRiP. In addition, 75% (n = 42) of clients achieved an ‘average’, ‘improved’, or ‘good’ school behaviour marking, while 86% (n = 47) of clients ‘improved’ their overall school performance. Finally, 66% (n = 56) of clients supported by TRiP demonstrated a reduction in ‘observable vulnerability’, while 82% (n = 121) of clients supported by TRiP experienced a moderate or strong reduction in ‘measured aggregate vulnerability’. 9.2 PRIMARY DATA COLLECTION RESULTS SUMMARY Primary data gathered through surveys and interviews provided a rich and descriptive understanding of TRiP’s impact on clients, caregivers, and human service providers. Exit surveys to children in the 11UI cohort reported that children felt ‘helped’ by TRiP. Child respondents also consistently reported ‘self improvement’, ‘positive changes in relationships with adults’, and ‘improved family life’. Exit surveys to youth in the twelve&up cohort self-reported ‘positive changes in behaviour’, ‘improved relationships’, ‘strengthened family communications’, and ‘more effective problem solving’. Caregivers of these two cohorts indicated that they felt ‘very supported’, were able to engage in services that they ordinarily would not have had, and were assisted in overcoming difficult barriers to pro-social activities and


professional supports. Within the caregiver exit data, some quantitative results were available. These results indicate that 85% (n = 35) of caregiver respondents were ‘satisfied’ or ‘very satisfied’. In addition, 90% (n = 37) of caregivers reported TRiP to have had a positive impact on their children, while 61% (n = 25) reported TRiP to have had an impact on their own parenting. Finally, caregiver exit survey data reveal that caregivers may have been challenged by TRiP support for their child ending, the large number of meetings, staff turnover, and the general shortness of a support window from TRiP. Results from the analysis of feedback to the Steering Committee Survey suggest that TRiP may be an evolutionary model in the human service field. Dialogue among this key group of stakeholders also describes TRiP has having the potential to change the community safety and well-being landscape in Saskatchewan. Some additional benefits of TRiP from the Steering Committee perspective include increased service provider capacity, improved inter-agency relationships, and better resolve to meet client needs. Additional observations are that TRiP has stabilized a hard-to-engage target group, enhanced service provider-client relations, increased collaboration among care providers, and has built a better understanding of client need. Some of the challenges identified by this cohort include the lack of sustainable funding, long wait times, an abrupt closure process, and home agency demands. Results from the surveys completed by C4 participants reveal a positive experience for human service providers. TRiP was described as having the ability to engage hard-to-reach clients, build valuable relationships, increase efficiency in service delivery, reduce barriers to information sharing, reduce service barriers for clients, and built trust between service providers and clients. Additional C4 dialogue indicates that service mobilizations within the model have been effective because TRiP staff collaborate to maintain momentum of the case plan. Furthermore, shared ownership has allowed for more effective solutions to support clients. Some of the outcomes that C4 participants attribute to TRiP include parent engagement, improved client attitude, and family connectedness. One unique challenge identified by C4 participants was that the effective support of TRiP for vulnerable families has inadvertently created a challenge of over-dependence on service providers. Finally, results from interviews with TRiP staff concur with results from other respondent cohorts in several areas. According to staff, TRiP coordinated support has contributed to improved client behaviour, enhanced commitment to school, elevated client confidence, and increased school attendance. From a parenting perspective, staff feel that TRiP has helped to improve parent involvement, build parent ownership over family issues, and increase client confidence and trust in working with human service providers. Concerning frontline service providers, interview results suggest that a multi-sector collaborative approach broadens perspectives on client needs and allows for more effective engagement of children and youth. With respect to human service agencies, staff respondents felt that TRiP improves unity among human service partners, reduces role confusion, improves collective effectiveness, and increases agency accountability for client outcomes. Additional results from TRiP staff interviews suggests that being located in the same building increases information sharing, fosters collaborative solution-building, and improves staff ability to monitor client needs. Also, by building relationships with many different organizations in Regina, TRiP has managed to mobilize strong and effective support networks around vulnerable children and families in the community. Finally, in providing reflection on the evaluation process, TRiP staff felt that participating in this evaluation has advanced and strengthened their own internal processes, which has led to increases in client needs identification, information sharing, and integrated support planning.


10.0 OVERALL FINDINGS The findings of this evaluation inform us on many different aspects of TRiP. To simplify our understanding of these aspects, the overall findings of this evaluation have been organized into 6 different themes. These themes, derived from the original evaluation questions driving this process, include: target group, process, satisfaction, efficiency, effectiveness, and outcome. 10.1 TARGET GROUP Within the evaluation period, TRiP has been able to engage a hard-to-reach client group, mainly consisting of male children and youth between the ages of 5 to 14. The vulnerability of this achieved target group was most-often attributable to the composite risk occurrences of anti-social behaviour, criminal involvement, school disengagement, mental health issues, physical violence, and parenting concerns, among other risk factors. An additional characteristic of this target group was difficulty overcoming personal, financial, and systemic barriers to services, supports, and pro-social activities in the community. 10.2 PROCESS Over the past few years, TRiP has largely been a development in progress. As more partners have committed to the initiative, more constructive discussions on the integrated support planning process occurred. Within this evaluation process, a robust examination of staff needs, partner capacity, and client support requirements led to a significant streamlining of process, practice, and internal reporting. In its most recent configuration, TRiP has emerged as a social innovation that fosters mutual accountability, shared client ownership, consistent service delivery, and multi-sector integrated support. Benefits Results of this evaluation reveal several benefits to multiple cohorts. Clients have benefited from increased service access, support in barrier reduction, and engagement in community activities. Similarly, integrated supports have helped parents feel supported, engage in appropriate services, and have benefitted from a unified support team. Through their collaborative experience in this initiative, both TRiP staff and human service providers have increased their understanding of client needs, built relationships required to gain quicker service access for their clients, and found more efficient and effective ways to problem solve. Finally, human service agencies have managed to build their collective capacity for generating client outcomes and have improved inter-agency relationships required for effective multi-sector coordinated support. Challenges Of course, TRiP has not been implemented in isolation of a few challenges. Some of these challenges have been mitigated or overcome, while others continue to remain a difficulty for TRiP staff. Combining results from interviews, surveys, internal reporting tools, and evaluator observations, the following challenges were detected within the evaluation period:

• Lack of sustainable, committed funding.

• Long waitlists fostered by increasing popularity, high client demands and limited capacity.


• Difficulty engaging parents to the level required for success.

• Home agency caseload demands.

• Lengthy intake and assessment process.

• Alignment of schedules for multiple service providers from different agencies.

• Variation in client-engagement practices among the partner agencies.

• Disparity in time commitments among TRiP partners.

• Difficulty for low-capacity organizations to become involved in custom coordinated case conferences.

• Limited options for TRiP when clients refuse to engage or do not show up.

Key Ingredients

One of the main reasons behind this evaluation was to not only assess the process and measure the outcomes of TRiP, but provide some knowledge to other communities looking to replicate TRiP. Based upon the results of this evaluation, a number of key ingredients are important for multi-sector coordinated support:

• Innovative thinking and pragmatic problem solving.

• Shared ownership over the initiative and client outcomes.

• Integrated/shared office facilities.

• Sustainable financial commitments.

• Sufficient human resource commitments from the partner agencies.

• Balanced representation of the various human service sectors.

• Strong inter-agency relationships.

• Solid, clear, and consistent process for needs identification, service mobilization, and ongoing coordinated support.

• Rigorous yet efficient internal reporting structure used to gather information and plan and prepare for multi-sector coordinated support.

• Strong backbone support for the partnership (e.g. administrative, coordination, strategic duties)

Future Considerations Throughout this evaluation process, qualitative feedback from multiple cohorts revealed several potential directions for TRiP moving into the future. Some of these included involving other agencies in TRiP’s Integrated Support Team—particularly those from the areas of culture, cognitive development, parenting support, addictions, and recreation. Another suggestion was for a more balanced division of labour among TRiP staff, particularly during the intake and assessment phase of a client’s involvement. A third consideration is for TRiP to empower C4 teams to absorb some of the coordination and logistics work once a solid team is established. This would allow TRiP to take on more clients and provide broader service supports.


10.3 SATISFACTION Overall, results from this evaluation indicate strong satisfaction with the initiative by all respondent groups. Limited data from clients themselves, suggest strong satisfaction with TRiP. In addition, caregiver and human service provider feedback not only indicates client satisfaction, but reveals that each of these respective groups are also satisfied with TRiP. Similarly, TRiP staff and Steering Committee members also report satisfaction with the initiative. Some of the main sources of satisfaction among all respondent groups include the strong organization of the TRiP process, consistency, seamless communication, a client-centred approach, strong team rapport, and an ability to help clients overcome barriers to services, supports, and activities. 10.4 EFFICIENCY The results of this evaluation indicate that TRiP has increased efficiency in some areas while being less efficient in other areas. On the one hand, quantitative and qualitative data demonstrate that clients have gained access to services through TRiP. Survey feedback indicates that this service access may not have happened so quickly (if at all) without TRiP’s coordinated support, thereby increasing agency efficiency in helping clients. On the other hand, the intake and assessment process—including the parent interview and initial Integrated Collaboration Team meeting, can lengthen some of the efforts made to help vulnerable children and youth. 10.5 EFFECTIVENESS Results of the evaluation show that TRiP has been effective in reducing barriers and connecting clients to service, supports, and activities. Quantitative and qualitative data show that TRiP has identified client needs, reduced barriers, mobilized appropriate services, and coordinated ongoing support to clients and their families. Exit surveys from both clients and caregivers report a feeling of clients being supported. Similarly, feedback from C4 participants report feeling supported by TRiP in finding ways to work collaboratively to meet the composite needs of shared clients. Remaining challenges in the area of effectiveness include reducing client refusals of service and caregiver/client lack of interest. 10.6 OUTCOMES This evaluation has measured outcomes of TRiP in several ways. Internal reporting, supported by data from partner agencies, reveal reductions in client risk; improvements in school attendance, behaviour, and performance; increased community engagement; and an overall reduction in aggregate vulnerability. Primary data gathered through surveys and interviews confirm that at least from the perspective of clients, caregivers, and human service providers—client behaviour, family connected-ness, and overall vulnerability have been reduced. Some contributors to this include increased service access; support in overcoming barriers to services, supports, and activities; and service providers building trusting relationships with vulnerable children/youth.


11.0 LIMITATIONS A common practice in evaluation and other social science research is to declare limitations in methodology and findings. Doing so allows readers to gather a complete sense of the study, including potential challenges in replication and/or generalization. One group of limitations common in many evaluations that involve qualitative data collection concern different types of biases. Despite a commitment to confidentiality and anonymity in this evaluation process, respondents may have been more likely to provide positive feedback because they received support/opportunities from TRiP. Furthermore, respondents to the various data collection instruments/interviews who were pleased with their TRiP experience may have been more likely to provide positive feedback than those who had a less positive experience. Finally, there is a chance that those who had a positive TRiP experience were more likely to complete an evaluation instrument than those who had a less positive experience. Another limitation in this evaluation is that in order to maximize available data, three types of TRiP client files were included in this analysis. One type included those who were clients of TRiP before the new reporting process was implemented, those who became clients of TRiP during implementation of the new reporting process, and those who became clients of TRiP after the reporting process was implemented. The result, in some cases, was incomplete data for clients who were involved in TRiP earlier on, as well as for those who became involved in TRiP later in the evaluation period. The consequence to this incomplete data was a smaller sample from an already small population (see paragraph below). One of the biggest limitations in this evaluation was the small population (n = 148) of TRiP clients who fully engaged. Although a majority of results were reported against the full assessment of all accepted TRiP clients (N = 360), assessing the evaluation variables for those clients who fully engaged in TRiP (i.e., completed at least one C4 meeting) was more important. Unfortunately, however, having such a small population of engaged clients made more advanced quantitative methodology difficult to pursue. A related challenge to the small sample of engaged clients is the potential that those clients who did engage, were more likely to trigger positive reporting (e.g. change in behaviour, stability) than those who did not engage. By making a commitment to engage in TRiP, it would not be unrealistic to expect these clients/families to have a higher probability of success than those who did not engage. Despite these limitations, one of the main strengths of this evaluation is that a diverse methodology has been deployed. Such diversity does allow for an examination of the evaluation questions in different ways. As the results suggest, quantitative analysis of internal reporting, combined with qualitative analysis of feedback from staff, clients, caregivers, and stakeholders do reveal a number of common themes.


12.0 CONCLUSION This evaluation of The Regina Intersectoral Partnership has explored several topics, many of which are relevant to TRiP stakeholders, as well as to other community safety and well-being stakeholders interested in multi-sector coordinated support. The findings of this 2-year evaluation process should build excitement and interest among human service agencies to explore and pursue TRiP-style innovations in human service coordination and delivery. In considering such explorations and pursuits, aspiring social innovators may wish to borrow from the many valuable lessons learned through TRiP’s experience. Not only does effective multi-sector coordinated support require partnership, but it requires equal commitment, shared ownership, and continuous monitoring of both challenges and successes. While it is beyond the scope of this evaluation to accurately predict the future of human service innovation in Canada, there is enough promise within this evaluation to suggest that multi-sector coordinated support may very well be a part of that landscape in the coming months and years. With regards to the results of this evaluation, due to the intensive nature of coordinated support facilitated by TRiP, the population size (n = 148) of the fully-engaged target group is limited. As such, the small N of clients examined in this evaluation places limitations on advanced quantitative analysis. However, as TRIP continues to build its client database, more analytical techniques will become available to evaluators. For now, this mixed methods examination of target group, satisfaction, process, efficiency, effectiveness, and outcomes, has informed us of the promising utility and strength of multi-sector collaboration in reducing risk and improving individual-level outcomes. As more data become available, future analyses of TRiP outcomes will be able to confirm the impact of vulnerability reduction on the broader state of community safety and well-being in Regina. Where TRiP itself is concerned, building upon past successes, refining current practices, and expanding the scope and reach of multi-sector coordinated support in Regina should front the agenda of upcoming activities for this collaborative. Strong partner support, coupled with new indications of vulnerability reduction, should give TRiP stakeholders the confidence required to retain sustainable funding, implement practical process improvements, and enhance the existing compliment of human service sectors contributing to TRiP’s outputs and outcomes. In close, this evaluation began with an intensive examination of the process and practices of TRiP. Through implementation of an internal reporting system; ongoing data collection; and inclusion of clients, caregivers, human service professionals, staff, and key stakeholders in the evaluation process; this evaluation grew to assess TRiP’s impact on client risk and vulnerability. While many more advanced opportunities for measuring these variables remain, the findings suggest that within the current evaluation period and achieved target group, TRiP has made a measurable impact on the safety and well-being of clients and families it has served through multi-sector coordinated support in Regina.


13.0 RECOMMENDATIONS To inspire further development and refinement of TRiP, parts of this evaluation were designed to identify opportunities to improve, strengthen, and expand the initiative. Based upon the results of this evaluation, combined with lessons learned from the broader community safety and well-being landscape in Canada, recommendations are proposed in three key areas: process, capacity, and configuration. In no particular order of importance, these recommendations offer both an action and rationale for TRiP stakeholders to consider in moving forward.

PROCESS 1) Examine the feasibility and partner interest in developing a capacity-building mechanism to support ongoing C4 support beyond TRiP involvement. Currently, a client will receive custom coordinated support so long as their TRiP file remains open. While the C4 partners may very well continue to collectively support their shared client beyond TRiP’s closure of that file, there is no formal mechanism to make that happen. As a result, many clients who could benefit from continued multi-sector collaboration are not guaranteed that opportunity. In the spirit of getting Regina human services more involved in collaboration (beyond just their TRiP representative) it may be beneficial to provide a process or protocol for C4 partners to continue supporting clients beyond the coordinating window of TRiP. 2) Construct and adopt a mechanism for ongoing detection and reporting of systemic issues, and where possible, collaborative solution building. Much of the work TRiP is occupied with involves reducing barriers to services, supports, and activities. In doing this work, TRiP staff are in a unique position to identify and report on barriers stemming from the current design, configuration, and capacity of the human service system. Furthermore, collaborative insight from a frontline multi-sector perspective may generate potential solutions for higher level decision-makers to either consider, or investigate further. 3) Condense the presentation format of new clients during Intersectoral Collaboration Team meetings. To accommodate more collaborative planning and service engagement, presentations of new clients at ICT meetings should be limited to providing sufficient information on client needs and risk to allow for a proper identification of appropriate services. The remaining more detailed information should be reserved for discussion among the case lead and C4 team members.

CAPACITY 4) Secure full-time commitments of staff from partner agencies. This evaluation has highlighted the potential to serve clients more efficiently in a collaborative environment. To maximize the impact of TRiP, and in-turn reduce demand of human service agencies in the long-term, staff compliments from the various partner agencies should be full-time. With such a commitment, come many options for improving capacity, reach, and intended outcomes. 5) Expand the current compliment of human service sectors represented at TRiP to involve full-time commitments from the following sectors: culture, cognitive development, housing, parenting support, addictions, and recreation. The current TRiP staff team has shown that multi-sector coordinated support, as a model for integrating human service delivery, is generating favorable results. Involving


these additional sectors would improve TRiP’s overall reach, performance, and capacity to meet the composite needs of clients and their families. 6) Develop the required formal partnerships and/or internal capacity to incorporate and provide enhanced mentoring support to TRiP clients. Although limited, one of the most valuable benefits of TRiP to clients and their families has been mentoring and support provided by TRiP staff. Considering the limitations of staff to provide additional mentoring without sacrificing core duties and responsibilities, there would be added value in TRiP either collaborating with a pre-existing mentor asset in Regina, or building capacity to offer the types of child/youth mentoring that caregivers and other human service providers recognize as being critical to sustainable vulnerability reduction among TRiP clients.

CONFIGURATION 7) Depart from the distinction and branding of 11UI and twelve&up initiatives. Despite the legacy of 11UI and twelve&up, there is little value in maintaining the distinct branding of these initiatives. In earlier years, as TRiP was transitioning from Stop Now and Plan toward a more integrated support program model, these titles were relevant. However, considering the current scope and direction of the initiative (e.g. less programmatic and more coordination-based), a unified TRiP identity for multi-sector coordinated support would be more appropriate and strategic. 8) Transition into a broad-sweeping multi-sector coordinated support process that is not limited by age, but is rather defined by family need. The evaluation data reported herein demonstrate that many of the risk factors placing children/youth in situations of vulnerability are just as much (if not more) attributable to caregivers (adults) than the younger clients themselves. With a slight realignment of the target group and referral process, TRiP could retain nearly all of its existing processes, while expanding its reach, accessibility, and quite potentially its longer-term outcomes. 9) Prepare and pursue a mobilization of new and existing project partners to implement the Hub Model of Collaborative Risk-Driven Intervention in Regina. Although already part of the broader strategic plan for TRiP, the immediate addition of a venue for ongoing risk detection, limited information sharing, and rapid intervention would be a perfect—if not necessary—compliment to the progress already being made by TRiP partners. While TRiP’s current process of ongoing multi-sector coordinated support has been shown to lower vulnerability among chronic higher risk families, a rapid triage solution would build the capacity for TRiP partners to mitigate risk earlier on, particularly among families who would be considered ‘newly at-risk’. An added benefit to mobilizing a Regina Hub through TRiP is that a majority of the groundwork for multi-sector partnerships, information sharing, and shared outcomes is already in place. In fact, communities outside Regina (both on-reserve and off-reserve) have already adopted a TRiP-inspired dyad of collaborative risk-driven intervention (e.g. Hub) and multi-sector coordinated support (e.g. C4). Most importantly, to avoid creating a new silo of collaboration in Regina, the Hub Model should be implemented and supported with TRiP as a backbone, while both new and existing partners form a larger Systems Leadership Group to guide these initiatives. 10) Expand the identity and role of TRiP to become a whole-of-system catalyst for collaboration in Regina. Traditionally, TRiP has been viewed as a multi-sector team that supports some of the most vulnerable children and youth in Regina. This has allowed for TRiP to evolve into one of a single entity having its own client base. With broader community collaboration and upstream risk mitigation in mind, TRiP should expand this identity and role beyond just collaboration for the purposes of supporting TRiP clients, and toward fostering collaboration among service providers for all vulnerable clients in Regina.


As a steward of multi-sector collaboration, with a quickened process of needs identification, information sharing, and service mobilization, TRiP could become a central catalyst of multi-sector coordinated support for all agencies in Regina.


REFERENCES Addiction and Mental Health Collaborative Project Steering Committee. (2015). Collaboration for Addiction and Mental Health Care: Best Advice. Ottawa, ON: Canadian Centre on Substance Abuse. Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J. A., Gendreau, P., & Cullen, F. T. (1990). Does Correctional Treatment Work? A clinically-relevant and psychologically-informed meta-analysis. Criminology, Vol 28(3): 369-404. Augimeri, L. K., Farrington, D. P., Koegl, C. J., & Day, D. M. (2007). The SNAP Under 12 Outreach Project: Effects of a community based program for children with conduct problems. Journal of Child and Family Studies, Vol 16(6): 799-807. Babayan, A., Landry-Thompson, T., and Stevens, A. (2015). Evaluation of the Brant Community Response Team Initiative: Six-month Report. Brantford, ON: Brant County Health Unit. Belleville Police Service. (2007). Police Interactions with Emotionally Disturbed/Mentally Ill People: A comprehensive analysis and review. Belleville, ON: Belleview Police Service. Bellmore, S. (2013). The Reentry of Young Offenders: A look at successful reintegration. Hamilton, ON: McMaster University, School of Social Work. Retrieved on 2017-05-22 from https://macsphere.mcmaster.ca/bitstream/11375/13731/1/fulltext.pdf. Bhayani, G., & Thompsom, S. K. (2016). SMART on Social Problems: Lessons learned from a Canadian risk-based collaborative intervention model. Policing: 1-16, Oxford University Press. Bonta, J., Wallace-Capretta, S., Rooney, J., & McAnooy, K. (2010). An Outcome Evaluation of a Restorative Justice Alternative to Incarceration. Contemporary Justice Review, Vol 5(4): 319-338. British Columbia Corrections Research Unit. (2009). The Impact of Respectful Relationships and Relationship Violence Treatment Programs on Spousal Assault and Recidivism: Research report. Vancouver, BC: Ministry of Public Safety and Solicitor General. Retrieved on 2017-05-22 from http://www2.gov.bc.ca/assets/gov/law-crime-and-justice/criminal-justice/corrections/research- evaluation/resp-relationship.pdf. British Columbia Ministry of Public Safety. (2011). Integrated Offender Management (IOM): The case for working together. Revealing Research and Evaluation (RREv), Issue 5. Retrieved on 2017-05-22 from http://www2.gov.bc.ca/assets/gov/law-crime-and-justice/criminal- justice/corrections/research-evaluation/issue-5.pdf. Chandrasekera, U., & Pajooman, N. (2011). Police & Mental Health: A critical review of joint police/mental health collaborations in Ontario. Toronto, ON: Provincial Human Services and Justice Coordinating Committee.


Cherner, R., Aubry, T., Kerman, N., & Nandlal, J. (2014). Transitioning into the Community: Outcomes of a pilot housing program for forensic patients. International Journal of Forensic Mental Health, Vol 13(1): 62-74. City of Calgary. (2010). 2009 PSC Operational Review Final Report. Calgary, AB: Emergency Services Consulting International. Retrieved on 2017-05-25 from https://www.calgary.ca/CSPS/PSC/Documents/PSC-911/operations-review- report.pdf?noredirect=1. Centers for Disease Control and Prevention. (2017). Child Abuse and Neglect: Risk and Protective

Factors. Atlanta, GA: CDC Injury Centre. Clairmont, D., & Waters, K. (2015). The Nova Scotia Restorative Justice Program: Assessment of current status and future directions. Halifax: NS: Dalhousie University Atlantic Institute of Criminology. Clement, L. (2016). Multiagency Early Risk Intervention Tables(s): Pilot Project Review. Ottawa, ON: Lansdowne Consulting Group. Commonwealth Department of Health and Aged Care. (2000). Promotion, Prevention, and Early

Intervention for Mental Health—A Monograph. Canberra, AUS: Author. Cooper, G. (2014). Evaluation of the Youth At Risk Development (YARD) Program: Final report. Calgary, AB: The Calgary Police Service. Retrieved on 2017-05-26 from http://calgarypolicefoundation.com/wp-content/uploads/2014/07/YARD_2014-Final- Eval_ExecSum_11_03_2014_kp.pdf. Cooper, H., Moore, L., Gruskin, S. & Krieger, N. (2005). The Impact of a Police Drug Crackdown on Drug Injectors’ Ability to Practice Harm Reduction: A qualitative study. Social Science & Medicine, Vol 61(3): 673-677. Correctional Service Canada. (2008). Evaluation Report: Correctional Service Coordination Agreement (Exchange of Services Agreement between CSC and New Brunswick Department of Public Safety). Fredericton, NB: New Brunswick Department of Public Safety. Correctional Service Canada. (2012). Federal Community Corrections Strategy: Vision to 2020. Ottawa, ON: Correctional Service Canada. Dumaine, F. (2005). Future Directions in Community Policing: Evaluation of the Ottawa Police Service Community Police Centres. The Canadian Review of Policing Research, Vol 1. Retrieved on 2017- 05-24 from http://crpr.icaap.org/index.php/crpr/article/view/43/49. Farnik, M., & Pierzchala, W. (2012). Instrument development and evaluation for patient-related outcomes assessments. Patient Related Outcome Measures, Vol. 21(3): 1-7. Fischer, D. G., & Jeune, R. (1987). Juvenile Diversion: A process analysis. Canadian Psychology, Vol 28(1): 60-70.


Fitch, D., & Grogan-Kaylor, A. (2012). Using Agency Data for Evidence-Based Programming: A university- agency collaboration. Evaluation and Program Planning, Vol. 35: 105-112. Gee, S. B., Croucher, M. J., & Beveridge, J. (2010). Measuring Outcomes in Mental Health Services for Older People: An evaluation of the Health of the Nation Outcome Scales for elderly people. International Journal of Disability, Development and Education, Vol. 52(2): 155-174. Giwa, S. (2008). Community Policing—A Shared Responsibility: Final project report and evaluation. A report prepared for the Department of Canadian Heritage, Multiculturalism Program. Ottawa, ON: South-East Ottawa Community Health Centre. Gossner, D., Simon, T., Rector, B., & Ruddell, R. (2016). Case Planning and Recidivism of High Risk and Violent Adult Probationers. Journal of Community Safety and Well-Being, Vol 1(2): 32-43. Government of Canada. (2015). Fredericton Police Force/St. Mary’s First Nation Partnership (Synopsis). Fredericton Police Force. Fredericton, NB: Public Safety Canada. Retrieved on 2017-05-22 from https://www.publicsafety.gc.ca/cnt/cntrng-crm/plcng/cnmcs-plcng/ndx/snpss-en.aspx?n=478. Gray, S. (2016) Lanark County Situation Table Project: Annual Report, September 2015 – September 2016. Perth, ON: Lanark County Situation Table. Hornick, J. P., Boyes, M., Tutty,L., & White, L. (2005). The Domestic Violence Treatment Option (DVTO), Whitehorse, Yukon: Final evaluation report. Ottawa, ON: National Crime Prevention Centre. Hubberstay, C., Rutman, D., & Hume, S. (2014). Evaluation of a Three-year Youth Outreach Program for Aboriginal Youth with Suspected Fetal Alcohol Spectrum Disorder. International Journal of Alcohol and Drug Research, Vol 3(1): 63-70. Huffman, L., Mehlinger, S., and Kerivan, A. (2000). Risk Factors for Academic and Behavioural Problems

at the Beginning of School. Fontana, WI: CE Learning Systems. Kerr, T., Small, W., & Wood, E. (2005). The Public Health and Social Impacts of Drug Market Enforcement: A review of the evidence. International Journal of Drug Policy, Vol 16: 210-221. Kirst, M., Pridham, N. R., Matheson, F., Young, L., Neidra, K., & Stergiopoulos, V. (2015). Examining Implementation of Mobile, Police-Mental Health Crisis Intervention Teams in a Large Urban Center. Journal of Mental Health, Vol 24(6): 369-374. Korb, K. A. (2012). Conducting Educational Research: Identify key variables and research design. Retrieved on 2017-07-12 from www.korbedpsych.com/R02Variables.html. Krameddine, Y. I., Demarco, D., Hassel, R., & Silverstone, P. H. (2013). A Novel Training Program for Police Officers that Improves Interactions with mentally Ill Individuals and is Cost-Effective. Frontiers in Psychiatry, Vol 4: 9-17. Lafortune, D. (2015). Parcours: Developing a strategy for preventing recidivism in the context of community corrections. Quebec, ON: Pratiques Psychologiques, Vol 21: 275-292.


Latimer, J., Dowden, C., & Muise, D. (2001). The Effectiveness of Restorative Justice Practices: A meta- analysis. Research Report RR2001-6E. Ottawa, ON: Research and Statistics Division, Justice Canada. Retrieved on 2017-05-23 from http://www.justice.gc.ca/eng/pi/rs/rep- rap/2001/rp01_1-dr01_1/p1.html. Lipman, E. L., Kenny, M., Sniderman, C., O’Grady, S., Augimeri, L., Khayutin, S., & Boyle, M. H. (2008). Evaluation of a Community-Based Program for Young Boys At-Risk of Antisocial Behaviour: Results and Issues. Journal of the Canadian Academy of Child and Adolescent Psychiatry, Vol 17(1): 12-19 Litchmore, R. (2015). Preliminary report on the progress of the Guelph Situation Table. Internal report provided to Guelph Situation Table. Guelph, ON: University of Guelph. Newman, F. L., Hunter, R. H., & Irving, D. (1987). Simple Measures of Progress and Outcome in the Evaluation of Mental Health Services. Evaluation and Program Planning, Vol. 10: 209-218. Nilson, C. (2015). The Original Game Changers: An Evaluative Report on Prince Albert’s Centre of Responsibility and its Role in the Advancement of Community Mobilization Efforts to Improve Community Safety and Wellness. Saskatoon, SK: Centre for Forensic Behavioural Science and Justice Studies – University of Saskatchewan. Orwin, R. G. (2000). Methodological Challenges in Study Design and Implementation: Assessing program fidelity in substance abuse health services research. Addiction, Vol. 95(S3): 9-27. Public Safety Canada. (2014). 2013-2014 Evaluation of the Aboriginal Community Safety Development Contribution Program: Final report. Ottawa, ON: Government of Canada. Retrieved on 2016-05- 26 from https://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/vltn-brgnl-cmmnty-sfty-2013-14/vltn- brgnl-cmmnty-sfty-2013-14-en.pdf. Rezansoff, S., Moniruzzaman, A., & Somers, J. M. (2013). An Initiative to Improve Outcomes Among Prolific and Priority Offenders in Six British Columbia Communities: Preliminary analysis of recidivism. Burnaby, BC: Simon Fraser University, Faculty of Health Sciences. Renger, R., Foltysova, J., Becker, K. L., & Souvannasacd, E. (2015). The Power of the Context Map: Designing realistic outcome evaluation strategies and other unanticipated benefits. Evaluation and Program Planning, Vol. 52: 118-125. Ringwalt, C., Ennett, S. T., & Holt, K. (1991). An Outcome Evaluation of DARE (Drug Abuse Resistance Education). Health Education Research Theory and Practice, Vol 6: 327-337. Roberts, A. R., & Yeager, K. R. (eds.). (2004). Evidence-Based Practice Manual: Research and Outcome Measures in Health and Human Services. New York, NY: Oxford University Press. Royal Canadian Mounted Police. (2007). D.A.R.E. Drug Abuse Resistance Education: National Client Survey 2007. Ottawa, ON: Government of Canada. Retrieved on 2017-05-24 from http://dare.procon.org/sourcefiles/royal-canadian-mounted-police.pdf.


Rugge, T., Bonta, J., & Wallace-Capretta, S. (2005). Evaluation of the Collaborative Justice Project: A restorative justice program for serious crime. Ottawa, ON: Public Safety and Emergency Preparedness Canada. Rush, B. (2003). The Evaluation of Treatment Services and Systems for Substance Use Disorders. Journal of Psychiatry and Psychotherapy, Vol. 25(3): 393-411. Shader, M. (2004). Risk Factors for Delinquency: An Overview. Washington, DC: U.S. Department of

Justice. Slinger, E., & Roesch, R. (2010). Problem-Solving Courts in Canada: A review and a call for empirically- based evaluation methods. International Journal of Law and Psychiatry, Vol 33(4): 258-264. Taylor-Powell, E. (2009). Evaluation Designs Common in Measuring Outcomes, Quick Tips #36. University of Wisconsin-Extension, Madison, WI. Available at in PDF format at http://www.uwex.edu/ces/pdanded/resources/. TRiP. (2014). The 11 and Under Initiative: Conceptual Framework. Regina, SK: The Regina Intersectoral

Partnership. TRiP. (2016). The Regina Intersectoral Partnership: Functioning Process Document – Defining the History,

Process, and Current Practice of the 11 and Under Initiative (11UI) and twelve&up. Regina, SK: The Regina Intersectoral Partnership.

van der Meulen, E., Claivaz-Loranger, S., Clarke, S., Ollner, A., & Watson, T. M. (2016). On Point: Recommendations for prison-based needle and syringe programs in Canada. Toronto, ON: Ryerson University. Walker, S. G., & Walker, C. R. (1992). The Process of Developing a Program Evaluation of a Community Policing Initiative. The Canadian Journal of Program Evaluation, Vol 7(2): 25-42. Werb, D., Elliott, R., Fisher, B., Wood, E., Montaner, J., & Kerr, T. (2007). Drug Treatment Courts in Canada: an evidence-based review. HIV AIDS Policy Law Review, Vol 12(2-3): 12-17. Wilson, R. J., Cortini, F., & McWhinnie, A. J. (2009). Circles of Support and Accountability: A Canadian national replication of outcome findings. Sexual Abuse: A Journal of Research and Treatment, Vol 21(4): 412-430. Wright, K. (2014). 11 & Under Initiative (11UI) Evaluation, Final Draft 2013-2014. Regina, SK: The Regina

Intersectoral Partnership


APPENDICES Appendix A: Internal Reporting Tools (appear alphabetically)

• Agency Referral Form

• Coordinated Custom Case Conference Report

• Caregiver Intake Guide

• Caregiver Referral Form

• Case Closure Report

• Child and Youth Intake Guide

• Community Engagement Form

• Community Networking and Outreach Form

• ICT Action Plan

• Post-TRiP School Report (Catholic)

• Post-TRiP Student Report (Public)

• School Background Report (Catholic)

• School Background Report (Public)

• School Engagement Summary

• TRiP Risk-Based Needs Assessment Tool Appendix B: Primary Data Collection Tools (appear alphabetically)

• C4 Closing Participant Survey

• C4 Initial Participant Survey

• Caregiver Exit Survey

• Child Exit Survey

• Stakeholder Closing Survey

• Stakeholder Initial Survey

• TRiP Staff Closing Interview Guide

• TRiP Staff Initial Survey

• Youth Exit Survey


APPENDIX A:


AGENCY REFERRAL FORM

It is suitable to make a referral to TRiP’s 11UI or twelve&up initiative if you have detected behaviours or conditions that place a child/youth in a place of vulnerability. Elements to consider for a referral to TRiP are:

• The child/youth exhibits multifaceted behavioural challenges.

• The child/youth is showing or is affected by composite risk factors.

• Previous engagements in services have shown little progress in the child/youth.

• The child/youth has experienced personal, situational, and/or institutional barriers to services and support.

• After having explored other options, the referring agent considers TRiP to be the best option. IMPORTANT NOTE: By completing this form, you are acknowledging that you have spoken to the client about TRiP and have gained their understanding concerning an opportunity for coordinated support through The Regina Intersectoral Partnership’s 11UI or twelve&up initiatives. INITIAL HERE: _____

This form will be used to gather the limited amount of information needed on a client to pursue the intake process.

Referring Agent Name:

Phone:

Referring Agency:

Email:

Describe agency role/relationship with the client:

Duration of relationship:

Child’s Full Name:

Birthday:

Is child regularly attending school?

__ Yes __ No

Are parents involved in their child’s school? (e.g.

volunteer, support activities, encourage attendance)

__ Yes __ No

Child’s School: (if not attending,

indicate reason)

Grade:

Primary Caregiver Name:

Relationship:

Caregiver Address:

Caregiver Phone: Caregiver Email:


What risk categories are relevant to your referral of this individual to TRiP?

__ alcohol

__ drugs

__ gambling

__ mental health

__ cognitive impairment

__ physical health

__ suicide

__ self-harm

__ criminal involvement

__ crime victimization

__ other (explain):

__ physical violence victim

__ physical violence perpetrator

__ emotional violence victim

__ emotional violence perpetrator

__ sexual violence victim

__ sexual violence perpetrator

__ elderly abuse perpetrator

__ poor supervision

__ basic needs

__ missing school

__ parenting concerns

__ housing

__ poverty

__ negative peers

__ anti-social behaviour

__ unemployment

__ missing/runaway

__ threat to public safety

__ gangs

__ social environment

What are some of the concerns that have led you to make this referral? (please explain)

What efforts have been made for this child?

Has the child/youth encountered any personal, situational or institutional barriers to support/services? (e.g.

transportation, parental support, financial barriers)

That you are aware of, what agencies are CURRENTLY involved in providing services or supports to this

individual?

That you are aware of, what agencies have PREVIOUSLY been involved in providing services or supports to

this individual?

To be completed by TRiP Referral & Intake Officer Only

Date/time caregiver contacted:

Verbal consent given:

__ yes __no

Other Notes:


COORDINATED CUSTOM CASE CONFERENCE REPORT

__ 11UI

__ SEO

__ 12&Up

__ SEO

__ SEO Only

Client Name:

Birthday:

Case Lead:

School:

Client ID#: Grade: TRiP Sectors Involved:

___ public edu ___health

___ catholic edu ___ police

___ social services ___ justice


CONFERENCE #1 Date:

Location:

Outputs of Conference (e.g. actions, plans, decisions): Challenges Endured (e.g. difficulties, resistance):

C4 Agency Participants

Name Agency Contact

Caregiver Participants

Name Relationship Contact

Regrets: Child Present:

___ yes ___ no

Support provided to overcome barrier:


Outcome of barrier reduction support (engaged in services, activities, etc.):

Other Notes:

Agency/Participant: Next Step/Actions:

Additional Notes


CAREGIVER INTAKE GUIDE

This guide is to be used to support the TRiP Intake Coordinator in conducting an intake interview with the candidate’s caregiver. Assigned TRIP ID: ______________

FAMILY INFORMATION Child’s Full Name:

Birthday:

Gender:


Relationship:

Primary Caregiver Address:

Caregiver Phone: Caregiver Email:

Secondary Caregiver Name:

Relationship:

Secondary Caregiver Phone: Caregiver Email:

Sibling 1 Name:

School:

Age:

Gender:

Sibling 2 Name:

School:

Age:

Gender:

Sibling 3 Name:

School:

Age:

Gender:

Sibling 4 Name:

School:

Age:

Gender:

What is child’s current living situation? (where, who, etc.)

SCHOOL INFORMATION Child’s School:

Grade: Teacher:


How often does your child attend school?

___ every day all day

___ once a day

___ every week

___ every month

___ few times a year

___ does not attend school

Do you feel you are engaged in your child’s school?

(please explain)

How does your child get to school?

___ walks alone

___ walks with siblings/friends

___ walks with caregiver

___ driven by caregiver

___ driven by city bus

___ driven by school bus

___ driven by taxi

___ other

How do you think your child feels about school?

___ enjoys it

___ is indifferent

___ does not like it

How do you feel about your child’s school?

___ happy with the school

___ not happy with the school (explain)

Does your child feel safe in school?

___ Yes ___No (explain)

Does your child feel safe on their way to school?

___ Yes ___No (explain)

If your child needs help with school work, who do

they go to?

How often do you or someone in your household help

your child with school work?

___ every day

___ every week

___ every month

___ a few times a year

___ never

Has your child ever been in trouble or experienced any particular issues in school? (explain)

Has your child ever received additional support or help in school? (explain)


HOME LIFE

How well does your child get along with everyone else in the home—including you, other adults, siblings, pets,

etc.?

What does a typical day look like in your household?

(weekdays)

(weekends)

What does your child like to do in their spare time?

What activities does your family do together?

COMMUNITY LIFE

What activities is your child involved in within the community?

What activities do you think your child wishes they could be involved in within the community?

What challenges/barriers has your child faced in trying to become involved in community activities?

What activities is your entire family involved in within the community?

THE CHILD/YOUTH

What are your child’s strengths?

What are your child’s challenges?


Has your child ever communicated that they wanted to improve something about his/herself? (explain)

Are there certain things that you think your child could improve upon? (explain)

Did you have any difficulties

with the pregnancy of this

child?

___ Yes ___ No

How often did you use alcohol

during your pregnancy?

___ never

___ occasionally

___ quite regularly

How often did you use drugs during

your pregnancy?

___ never

___ occasionally

___ quite regularly

Were there any delays in your child’s key milestones? (e.g. crawling, walking, talking)

___ Yes (explain) ___ No

Has your child ever witnessed a traumatic event? (explain)

Has your child ever suffered significant grief or loss? (explain)

Has your child ever witnessed or been exposed to any type of violence? (explain)

___ physical violence

___ psychological/emotional violence

___ sexual violence

Has your child ever received a diagnosis for cognitive impairment?


Has your child ever received a diagnosis for a mental health condition?


Is your child on any medication?


How does your child respond to authority?

How does your child handle conflict with the following cohorts?

Caregivers:


Authority:

Other Children:

Does your child make friends

easily?

___ Yes ___ No

Does your child show remorse or

take responsibility for his/her

actions?

___ Yes ___ No

Does your child have good self-

monitoring skills?

___ Yes ___ No

Has your child ever been inappropriate towards others?


Has your child ever been inappropriate towards animals?

___Yes (explain) ___ No

Do you have any general concerns about your child’s behaviour?(explain)

What, if any, concerns do you have regarding other aspects of your child? (explain)

THE FAMILY

What are your family’s overall strengths?

What are your family’s challenges?

What are the issues affecting your family? (explain)

Identify risk factors through conversation, not by direct inquiry?

__ alcohol

__ drugs

__ gambling

__ mental health


__ physical health

__ suicide

__ self-harm



__ physical violence

__ emotional violence

__ sexual violence

__ elderly abuse

__ supervision

__ basic needs

__ missing school

__ parenting

__ housing

__ poverty

__ negative peers


__ unemployment

__ missing/runaway


__ gangs



Who are your family’s key personal supports?

What would help your family improve its overall health and happiness?

SERVICE ENGAGEMENT

What current supports does your family access?

Has your family ever had challenges finding or accessing supports? (explain)

What services or supports do you think your family would benefit from?

___ social services

___ social assistance

___ housing

___ mental health

___ sexual health

___ public health

___ medical health

___ addictions

___ harm reduction

___ counselling

___ cultural support

___ spiritual support

___ parenting support

___ education support

___ employment support

___ home care

___ life skills

___ victim support

___ safe shelter

___ police

___ courts

___ corrections

___ probation

___ parole

___ legal support

___ fire department

___ mentorship

___ recreation

___ food support

___ other

COMMITMENT

Are you willing to make a commitment to engage in and support TRiP’s efforts to help your child?

___ Yes ___ No


CAREGIVER REFERRAL FORM

By completing this form, you are communicating to The Regina Intersectoral Partnership that you feel your child would be well served by the coordination of supports provided by partner agencies involved in either the 11UI or the twelve&up Initiative. This form does not substitute as consent to participate. An intake interview and follow-up communication among human service providers regarding your child will be required. Child’s Full Name:

Birthday:

School Attending: (if not attending,

indicate reason)

Grade:


Relationship:

Address:

Phone: Email:

What are some of the concerns about your child that have led you to make this referral?

Has your child received support for any of your concerns in the past? ___yes ___ no

If yes, please explain:

Is your child currently receiving support from any of the following services?

___ mental health ___ social services ___ school counsellor

___ addictions ___ corrections ___ other: ____________________________________

Please return completed form to the following TRiP sector representative:

_____________________________________ Or by Calling the Intake & Referral Officer at 306-523-3024


CASE CLOSURE REPORT

Client Name:

Birthday:

Case Lead:

Client ID: Grade:

TRiP Sectors Involved:



___ social services ___ justice School:

Main reason for referral:

Risk factors the C4 worked on:

__ alcohol

__ drugs

__ gambling

__ mental health


__ physical health

__ suicide

__ self-harm





__ sexual violence

__ elderly abuse

__ supervision

__ basic needs

__ missing school

__ parenting

__ housing

__ poverty

__ negative peers


__ unemployment

__ missing/runaway


__ gangs


Services that were provided:

___ social services


___ housing

___ mental health

___ sexual health

___ public health

___ medical health

___ addictions

___ harm reduction

___ counselling






___ home care

___ life skills

___ victim support

___ safe shelter

___ police

___ courts

___ corrections

___ probation

___ parole

___ legal support

___ fire department

___ mentorship

___ recreation

___ food support

___ other

Post-Support School Engagement

Attendance:

Performance:

Behaviour:

SUPPORT SERVICES

Services Connected:

Barriers to Services Overcome:


COMMUNITY ENGAGEMENT

Activities Engaged: Barriers to Activities Overcome:

FAMILY

Family concerns overcome:

Services provided to family members:

OBSERVABLE CHANGE OVERALL

Change in Conditions:

Change in Behaviour:

Indicators of Reduced Vulnerability:

Reason for Closure:

__ C4 closed it: stabilized __ refused consent __ unable to locate

__ C4 closed it: potential to stabilize __ refused services __ moved away

__ aged out __ not engaging __ in custody/care

__ screened out __ family not supportive __ other


CHILD & YOUTH INTAKE GUIDE

This guide is to be used to support the TRiP Intake Coordinator in conducting an intake interview with the child/youth for either 11UI or twelve&up. Assigned TRIP ID: ______________

PERSONAL Child’s Full Name:

Birthday:

Gender:

What are your personal interests (hobbies, favorite things, etc.)?

What are your personal strengths? What are your personal weaknesses?

What, if anything do you want to improve about

yourself?

Is there anything in your life that is bothering you?

What are your friends like?

What adults do you respect/admire?

What are the problems in your life that are affecting you right now?

What do you do when you have a problem?

Do you talk to anyone when you need help or you have a problem? (explain)

FAMILY Do you feel that you have a good relationship

with your parents? (explain)

Do you feel that you have a good relationship with your

siblings? (explain)


Do you listen to

your parents?

___ Yes

___ No

What happens when you do not listen to your parents?

SCHOOL Do you attend school regularly?

___ every day, all day

___ at least once a day

___ every week

___ every month

___ a few times a year

___ never

Do you like school?

___ Yes

___ No

Do you feel safe in school?

___ Yes

___ No

When you miss school, why do you miss school?

Do you have a good relationship with your teachers?

___ Yes

___ No

What do you think is challenging about school?

What do you like best about school?

Have you ever been in trouble or experienced any particular issues in school? (explain)

Have you ever received additional support or help in school? (explain)

SERVICES & SUPPORTS

Have you ever had any type of worker? (e.g. social worker, addictions worker, counsellor)

How would you characterize your relationship(s) with your past worker(s)?

Who is your “go-to” person when you are feeling scared or in trouble?

Who is someone you trust when you need help?


Have you ever faced any barriers or challenges in getting help or support?

COMMUNITY LIFE

What activities are you involved in within the community?

What makes it difficult for you to participate in community activities?

What activities do you wish you could be involved in within the community?

INDIVIDUAL RISK

Have you ever tried or used alcohol? ___ Yes ___ No ___ Unsure

Have you ever tried or used drugs? ___ Yes ___ No ___ Unsure

Have you ever been very sad for a long time? ___ Yes ___ No ___ Unsure

Have you ever been extremely stressed? ___ Yes ___ No ___ Unsure

Have you ever thought of committing suicide? ___ Yes ___ No ___ Unsure

Have you ever intentionally hurt yourself? ___ Yes ___ No ___ Unsure

Have you ever broken the law? ___ Yes ___ No ___ Unsure

Have you ever been a victim of violence? ___ Yes ___ No ___ Unsure

Have you ever been violent towards others? ___ Yes ___ No ___ Unsure

Have you ever felt that your basic needs were not met? ___ Yes ___ No ___ Unsure

Do you find that you argue with others a lot? ___ Yes ___ No ___ Unsure

Do you feel that your friends are a bad influence? ___ Yes ___ No ___ Unsure

Have you every ran away? ___ Yes ___ No ___ Unsure

Are you involved or have you ever been involved in a gang? ___ Yes ___ No ___ Unsure

Do you feel unsafe in your neighbourhood? ___ Yes ___ No ___ Unsure


COMMUNITY ENGAGEMENT FORM

This form shall be used to record community engagement support provided to clients of the 11UI or twelve&up initiative. This information will be used to track and monitor the community engagement outputs generated through TRiP. Staff Name(s): Date of Support:

Client Name:

Birthday: Initiative:

Client ID: __ 11UI

__ SEO

__ 12&Up

__ SEO

__ SEO

Only

Description of activity:

How was the activity planned (origin of idea, identification of interest, etc.)?

Observations of client’s reaction to activity:

Duration of activity:

___ one time activity

___ occasional activity

___ ongoing activity

Barriers to activity:

___ Personal (e.g., stigma, fear, distrust, denial, no time management)

___ Financial (e.g., can’t afford activity, no money)

___ Situational (e.g., child care, transport, affected by family risk factors)

___ Institutional (e.g., long wait-list, staff turnover, hours of operation)

___ other:

Transport Provided:

___ yes ___ no

Notes for client file:


COMMUNITY NETWORKING & OUTREACH FORM

This form shall be used by the TRiP staff to record any promotions of TRiP through presentations, major networking opportunities or unilateral outreach to other community agencies, government representatives or members of the public. Staff Name(s): Date of Outreach:

Outreach Target:

Origin of outreach opportunity:

Focus of Outreach: ___ 11UI ___ 12&UP ___ Hub ___ TRiP Overall ___ other

Format of Outreach: ___ formal presentation ___ sharing network ___ casual outreach

___ other:

Total # of human service professionals receiving

information: ____

Total # of general members of the public receiving

information: ___

Names of agencies receiving information through outreach:

Observed reactions of outreach audience:

Follow-up recommendations with audience:


INTERSECTORAL COLLABORATION TEAM (ICT) ACTION PLAN __ 11UI

__ SEO

__ 12&Up

__ SEO

__ SEO Only

Client Name:

Birthday:

Case Lead:

Client ID #: Grade:

TRiP Sectors Involved:



___ social services ___ justice Assessment Score: School:

Main reason for referral:

Referral Source:

Risk factors to work on:

__ alcohol

__ drugs

__ gambling

__ mental health


__ physical health

__ suicide

__ self-harm





__ sexual violence

__ elderly abuse

__ supervision

__ basic needs

__ missing school

__ parenting

__ housing

__ poverty

__ negative peers


__ unemployment

__ missing/runaway


__ gangs


Services to be engaged:

___ social services


___ housing

___ mental health

___ sexual health

___ public health

___ medical health

___ addictions

___ harm reduction

___ counselling






___ home care

___ life skills

___ victim support

___ safe shelter

___ police

___ courts

___ corrections

___ probation

___ parole

___ legal support

___ fire department

___ mentorship

___ recreation

___ food support

___ other

Barrier (s)Type: ___ personal ___ financial ___ situational ____ institutional ___ other

Description of barrier(s):

How was barrier(s) detected:


ICT Actions:

___ Accepted – Refer to C4

___ Rejected – defer to agency outside TRiP

___ Rejected – Make recommendations to caregivers

___ Other: __________________________________

Logistics to Consider (e.g. potential time, location, details, family availability)

Participants to invite to case conference:

Name: Agency:

Other Notes:


CATHOLIC SCHOOL POST-TRiP STUDENT REPORT

This report was designed to gather information from schools on children referred to the 11UI or twelve&up initiative. At this point, consent for information sharing among the TRiP partners has already been provided by the parent(s). The information requested through this report will help in the evaluation and assessment of student progress following their involvement in TRiP.

Trip Information (TRiP to enter)


Birthday:

Date of Referral: Date of first C4 Meeting:

SCHOOL INFORMATION (School to enter) School: School Representative:

PARENT ENGAGEMENT (School to enter) Since the student’s involvement with TRiP began,

are his/her caregivers engaged with the school?

___ No ___Yes (explain)

Since the student’s involvement with TRiP began, are

his/her caregivers engaged with the student’s learning?


STUDENT PERFORMANCE (School to enter) Since the student’s involvement in TRiP began,

how often is the student late for class?

___ never

___ rarely

___ several times a week

___ several times a month

___ every day


Since the student’s involvement in TRiP began, how

often is the student absent from class?

___ never

___ rarely



___ every day


Since the student’s involvement in TRiP began,

what is the student’s performance in literacy?

___ excels

___ meeting

___ beginning

___ not yet

___ incomplete

Since the student’s involvement in TRiP began, what is

the student’s performance in numeracy?

___ excels

___ meeting

___ beginning

___ not yet

___ incomplete


STUDENT ENGAGEMENT, SUPPORT & BARRIERS (School to enter) Since the student’s involvement in TRiP began,

has the student become engaged in school

activities?


Since the student’s involvement in TRiP began, has the

student become engaged in community activities outside

of school?

___ No ___ Yes (explain)


has the student been supported by human service

professionals?



student been able to overcome any barriers (e.g.

personal, financial, institutional)


BEHAVIOUR (School to enter) Since the student’s involvement in TRiP began, how would you describe the student’s behaviour in the

school?

___ good ___ average ___ poor

Since the student’s involvement in TRiP began, have you observed any changes in the student’s behaviour?

___ no ___ yes (please explain)


PUBLIC SCHOOL POST-TRiP STUDENT REPORT

This report was designed to gather information from schools on children referred to the 11UI or twelve&up initiative. At this point, consent for information sharing among the TRiP partners has already been provided by the parent(s). The information requested through this report will help in the evaluation and assessment of student progress following their involvement in TRiP.

Trip Information (TRiP to enter)


Birthday:

Date of Referral: Date of first C4 Meeting:

SCHOOL INFORMATION (School to enter) School: School Representative:

PARENT ENGAGEMENT (School to enter) Since the student’s involvement with TRiP began,

are his/her caregivers engaged with the school?


Since the student’s involvement with TRiP began, are

his/her caregivers engaged with the student’s learning?


STUDENT PERFORMANCE (School to enter) Since the student’s involvement in TRiP began,

how often is the student late for class?

___ never

___ rarely



___ every day


Since the student’s involvement in TRiP began, how

often is the student absent from class?

___ never

___ rarely



___ every day



what is the student’s performance in literacy?

___ established

___ meeting

___ progressing

___ beginning

___ insufficient

Since the student’s involvement in TRiP began, what is

the student’s performance in numeracy?

___ established

___ meeting

___ progressing

___ beginning

___ insufficient


STUDENT ENGAGEMENT, SUPPORT & BARRIERS (School to enter) Since the student’s involvement in TRiP began,

has the student become engaged in school

activities?



student become engaged in community activities outside

of school?

___ No ___ Yes (explain)


has the student been supported by human service

professionals?



student been able to overcome any barriers (e.g.

personal, financial, institutional)


BEHAVIOUR (School to enter) Since the student’s involvement in TRiP began, how would you describe the student’s behaviour in the

school?

___ good ___ average ___ poor

Since the student’s involvement in TRiP began, have you observed any changes in the student’s behaviour?

___ no ___ yes (please explain)


CATHOLIC SCHOOL BACKGROUND REPORT

This report was designed to gather information from schools on children referred to the 11UI or twelve&up initiative. At this point, consent for information sharing among the TRiP partners has already been provided by the parent(s). The information requested through this report will help in the determination of clients needs, vulnerabilities, risk factors, assets and community supports.

SCHOOL INFORMATION School: School Representative:

Title:

Email:

Phone:

STUDENT INFORMATION Child’s Full Name:

Birthday:

TRiP ID #: Gender:

Teacher: Grade:

School’s Main Contact for Student:

Relationship:

School’s Second Main Contact for Student:

Relationship:


Does the student have siblings that attend the same school?

___ Yes ___ No

Are the student’s caregivers engaged with the school?

(please explain)

Are the student’s caregivers engaged with the

student’s learning? (please explain)


PERFORMANCE How often is the student late for class?

___ never

___ rarely



___ every day


How often is the student absent from class?

___ never

___ rarely



___ every day


What is the student’s performance in literacy?

___ excels

___ meeting

___ beginning

___ not yet

___ incomplete

What is the student’s performance in numeracy?

___ excels

___ meeting

___ beginning

___ not yet

___ incomplete

Do you have any general observations of the student’s overall performance in school? Or what may affect

his/her performance?

ENGAGEMENT, ACCESS & SUPPORT Is the student engaged in school activities?


Is the student engaged in activities outside of school? ___ No

___Unsure ___ Yes (explain)

Has the student ever received additional support or help in school? (explain)

What challenges/barriers has the student faced in trying to become involved in support services?

What challenges/barriers has the student faced in trying to become involved in community activities?

What support services would the student benefit from?

What community activities would the student likely want to become involved in?


BEHAVIOUR How is the student’s behaviour in the school?

___ good

___ average

___ poor

Does the school have any behavioural concerns about

the student?

Does the student get along well with others?

___ yes ___ no

How would you rate the student’s social

skills?

___ good

___ average

___ poor

How would you rate the student’s conflict resolution skills?

___ good

___ average

___ poor

Has the student ever been in trouble or experienced any particular issues in school? (explain)

What are the student’s strengths?

What are the student’s weaknesses?

Please share any other observations or knowledge of the student that would help in the determination and

planning of multi-sector coordinated case management aimed at reducing the student’s overall vulnerability?


PUBLIC SCHOOL BACKGROUND REPORT

This report was designed to gather information from schools on children referred to the 11UI or twelve&up initiative. At this point, consent for information sharing among the TRiP partners has already been provided by the parent(s). The information requested through this report will help in the determination of clients needs, vulnerabilities, risk factors, assets and community supports.

SCHOOL INFORMATION School: School Representative:

Title:

Email:

Phone:

STUDENT INFORMATION Child’s Full Name:

Birthday:

TRiP ID#: Gender:

Teacher: Grade:

School’s Main Contact for Student:

Relationship:

School’s Second Main Contact for Student:

Relationship:


Does the student have siblings that attend the same school?

___ Yes ___ No

Are the student’s caregivers engaged with the school?

(please explain)

Are the student’s caregivers engaged with the

student’s learning? (please explain)

PERFORMANCE How often is the student late for class?

___ never

___ rarely



___ every day


How often is the student absent from class?

___ never

___ rarely



___ every day



What is the student’s performance in literacy?

___ established

___ meeting

___ progressing

___ beginning

___ insufficient

What is the student’s performance in numeracy?

___ established

___ meeting

___ progressing

___ beginning

___ insufficient

Do you have any general observations of the student’s overall performance in school? Or what may affect

his/her performance?

ENGAGEMENT, ACCESS & SUPPORT Is the student engaged in school activities?


Is the student engaged in activities outside of school? ___ No

___Unsure ___ Yes (explain)

Has the student ever received additional support or help in school? (explain)

What challenges/barriers has the student faced in trying to become involved in support services?

What challenges/barriers has the student faced in trying to become involved in community activities?

What support services would the student benefit from?

What community activities would the student likely want to become involved in?

BEHAVIOUR How is the student’s behaviour in the school?

___ good

___ average

___ poor

Does the school have any behavioural concerns about

the student?

Does the student get along well with others?

___ yes ___ no

How would you rate the student’s social

skills?

___ good

___ average

___ poor

How would you rate the student’s conflict resolution skills?

___ good

___ average

___ poor


Has the student ever been in trouble or experienced any particular issues in school? (explain)

What are the student’s strengths?

What are the student’s weaknesses?

Please share any other observations or knowledge of the student that would help in the determination and

planning of multi-sector coordinated case management aimed at reducing the student’s overall vulnerability?


SCHOOL ENGAGEMENT SUMMARY

This form shall be used by the TRiP School Engagement Officer to record ongoing updates of school engagement support provided to clients of the 11UI and twelve&up initiatives. School: Case Lead:

Initiative:

Client Name: __ 11UI

__ SEO

__ 12&Up

__ SEO

__ SEO Only

Client ID #: Birthday: TRiP Sectors Involved:



___ social services ___ justice

Type of school engagement concern: ___ lateness ___ absenteeism ___ both ___ other

Barriers to school engagement:

SCHOOL ENGAGEMENT ACTIVITIES DATE DESCRIPTION OF ACTIVITY OUTCOME

Additional notes for client file:

Collaboration involved:








TRiP Risk-Based Needs Assessment Tool

Through a risk review and scoring process, this tool is used to guide a needs assessment for the 11UI and twelve&Up initiatives. Professional discretion should always be the final judgement in needs assessment. However, the needs assessment score can be used as an indicator of need complexity and severity. Information for this tool must be gathered from 4 different sources: School Background Report, Child and Youth Intake Guide, Caregiver Intake Guide, and Agency Referral Form. Where a variant is present, simply apply the value for that variant in the score column. Then tally up the score and read interpretation offered below. Client Name: _________________________ Client ID#: _______________ Date: _________________________

DATA SOURCE VARIABLE VARIANTS VALUE SCORE

Agency Referral

Form (corroborate with Caregiver

Intake Guide)

Reported

Risk

alcohol 1

drugs 1

gambling 1

mental health 1

cognitive impairment 1

physical health 1

suicide 1

self-harm 1

criminal involvement 1

crime victimization 1

physical violence victim 1

physical violence perpetrator 1

emotional violence victim 1

emotional violence perpetrator 1

sexual violence victim 1

sexual violence perpetrator 1

elderly abuse perpetrator 1

poor supervision 1

basic needs 1

missing school 1

parenting concerns 1

housing 1

poverty 1

negative peers 1

anti-social behaviour 1

unemployment 1

missing/runaway 1

threat to public safety 1

gangs 1

social environment 1

Caregiver Intake

Guide

Life

Experience

delayed achievement of developmental milestones 1

experienced a traumatic event 1

experienced grief or loss 1

inappropriate towards others 1

inappropriate towards animals 1

Child & Youth

Intake Guide

Self-

Identified

Risk

tried or used alcohol 1

tried or used drugs 1

has been very sad for a long time 1

has been extremely stressed 1

has thought of committing suicide 1


has intentionally hurt self 1

has broken the law 1

has been a victim of violence 1

has been violent towards others 1

feels that basic needs are not met 1

argues with others a lot 1

feels that friends are a bad influence 1

has ran away 1

is or has been involved in a gang 1

feels unsafe in neighbourhood 1

Child & Youth

Intake Guide (corroborate with Caregiver

Intake Guide)

Community

Engagement

is currently involved 0

trying to become involved 1

not involved 2

School

Background

Report

School

Engagement

is currently engaged 0

is not currently engaged 2

School

Absence

is never absent 0

is rarely absent 0

absent several times a week 1

absent several times a month 2

absent every day 3

does not attend school 4

TOTAL NEEDS

SCORE

NEEDS ASSESSMENT INTERPRETATION

Needs Score Suggested Interpretation Suggested Action

0 to 5 could benefit from support connect to appropriate services

6 to 19 would benefit from support explore support options at Intersectoral Collaboration Team

20 or higher needs significant support prepare for Custom Coordinated Case Conference

* If you need to add a data point (e.g. risk factor) that was not in the original instrument (e.g. Referral Form), and you’re confident in the accuracy of your information, it is ok to add it. However, please mark that data point with an asterisk to identify that you added the observation of that data point separate from the original form.


APPENDIX B


C4 Closing Participant Survey As part of the evaluation of TRiP, participants in TRiP’s custom coordinated case conferences (C4) are asked to complete a confidential survey on their experience with TRiP to date. Your completion of this survey is completely voluntary. Choosing not to complete the survey will not impact your relationship with TRiP. Your answers will remain anonymous and confidential. The answers to the survey questions will only be seen and analysed by Dr. Chad Nilson of the Living Skies Centre for Social Inquiry. Results of the survey will be reported in the aggregate so that no sectors or individuals can be identified. Please send your completed surveys to Dr. Nilson at [email protected] by March 21st. 1) What is your overall perspective on your experience with TRiP? answer 2) What has the collaboration among TRiP partners done for your clients? answer 3) What has the collaboration among TRiP partners done for the participating agencies (like yours)? answer 4) What are some of the successes you have experienced with TRiP? answer 5) What have been some of the challenges you have experienced? answer 6) What are some opportunities for improving TRiP? answer 7) How well do you feel other agencies and professionals participate in the Custom Coordinated Case Conferences facilitated by TRiP staff? answer 8) What barriers to services and support has TRiP helped children, youth and family overcome? answer 9) Do you feel that TRiP has mobilized services around individuals? Please explain you answer. answer 10) Do you feel that TRiP has integrated services engaged with individuals? Please explain your answer. answer 11) What has been the observable impact of TRiP on clients served through this initiative? answer

Please email your response to Dr. Nilson at [email protected]. Thank you so much.


C4 Initial Participant Survey As part of the evaluation of TRiP, participants in TRiP’s custom coordinated case conferences (C4) are asked to complete a confidential survey on their experience with TRiP to date. Your completion of this survey is completely voluntary. Choosing not to complete the survey will not impact your relationship with TRiP. Your answers will remain anonymous and confidential. The answers to the survey questions will only be seen and analysed by Dr. Chad Nilson of the Living Skies Centre for Social Inquiry. Results of the survey will be reported in the aggregate so that no sectors or individuals can be identified. Please send your completed surveys to Dr. Nilson at [email protected] by April 28th. 1) What is your overall perspective on your experience at TRiP so far? answer 2) What has the collaboration among TRiP partners done for clients? answer 3) What has the collaboration among TRiP partners done for the participating agencies (like yours)? answer 4) What are some of the early successes you have experienced with TRiP? answer 5) What have been some of the challenges you have experienced? answer 6) What are some opportunities for improving TRiP? answer 7) How well do you feel other agencies and professionals participate in the Custom Coordinated Case Conferences facilitated by TRiP staff? answer 8) What barriers to services and support has TRiP helped children, youth and family overcome? answer 9) Do you feel that TRiP has mobilized services around individuals? Please explain you answer. answer 10) Do you feel that TRiP has integrated services engaged with individuals? Please explain your answer. Answer 11) Do you have any other observations or feedback? Answer



CAREGIVER SURVEY As part of our commitment to making sure we provide the best services possible to children and their families, we ask all caregivers to complete a survey. Your completion of this survey is voluntary. If you do not complete this survey, no harm or retribution will be caused to you. If you choose to complete the survey, your name and answers will remain confidential and anonymous. The results of these surveys will be used to identify progress and make improvements to the 11 and Under Initiative and the Twelve and Up Initiative. Thank you for taking the time to provide us with an account of your experience with TRiP.

1) What support did the TRiP team provide to your child and the rest of your family?

2) How satisfied were you with the delivery of this support?

___ not satisfied ___ somewhat satisfied ___ satisfied ___ very satisfied

3.a) Have you or your child ever accessed human service supports in the past? ___ Yes ___ No

3.b) If yes, was there any noticeable difference in the way you received services through TRiP’s support?

4) Did anything make it easy for you to access services?

5) Did anything make it difficult for you to access services?

6) How supported do you think your child felt in the program?

___ not supported ___somewhat supported ___ satisfied ___very supported

7) How supported did you feel in the program?

___ not supported ___somewhat supported ___ satisfied ___very supported

8.a) Has TRiP allowed you to do anything you were unable to do before? ___ Yes ___ No

8.b) If yes, please explain.


9.a) Have you noticed a change in your child since their time involved with TRiP? ___ Yes ___ No

9.b) If yes, please explain.

10.a) Have there been any changes in your own parenting because of TRiP? ___ Yes ___ No

10.b). If yes, please explain.

11) What suggestions do you have for improving the support that you and your family received through

TRiP?

12) Do you have any other comments about the 11 and Under or Twelve and Up Initiatives?

Please return completed survey to TRiP. Thank you so much.


CHILD EXIT SURVEY

To see how we are doing, we want to ask children to complete this survey. We won’t tell anyone your answers. Promise!! Thank you so much.

PLEASE WRITE YOUR ANSWERS TO EACH QUESTION



How have you become

better?

Is there anything different

about your family?

Has anyone helped you?


Stakeholder Closing Survey As part of the evaluation of TRiP, participants in TRiP’s key stakeholders are asked to complete a confidential survey on their experience with TRiP to date. Your completion of this survey is completely voluntary. Choosing not to complete the survey will not impact your relationship with TRiP. Your answers will remain anonymous and confidential. The answers to the survey questions will only be seen and analysed by Dr. Chad Nilson of the Living Skies Centre for Social Inquiry. Results of the survey will be reported in the aggregate so that no agencies, sectors or individuals can be identified. Please send your completed surveys to Dr. Nilson at [email protected] by March 21st. 1) As an agency leader, how would you describe your experience with TRiP? answer 2) What has the collaboration among TRiP partners done for your agency? answer 3) What are some of the successes you have observed with TRiP? answer 4) What have been some of the challenges you are aware of? answer 5) What are some opportunities for improving TRiP? answer 6) What has been the observable impact of TRiP on clients served through this initiative? answer 7) Has TRiP impacted your agency’s overall perspective on “community” and “collaboration”? If so, how? answer 8) What do you feel the “next steps” for TRiP should be moving forward? answer



Stakeholder Initial Survey As part of the evaluation of TRiP, partners to TRiP are asked to complete a confidential survey on their experience with TRiP to date. Your completion of this survey is completely voluntary. Choosing not to complete the survey will not impact your relationship with TRiP. Your answers will remain anonymous and confidential. The answers to the survey questions will only be seen and analysed by Dr. Chad Nilson of the Living Skies Centre for Social Inquiry. Results of the survey will be reported in the aggregate so that no sectors or individuals can be identified. 1) What is your overall perspective on your experience at TRiP so far? answer 2) What has the collaboration among TRiP partners done for clients? answer 3) What has the collaboration among TRiP partners done for the participating agencies? answer 4) What are some of the early successes you have experienced at TRiP? answer 5) What have been some of the challenges you have experienced? answer 6) What are some opportunities for improving TRiP? answer 7) How well have agencies and professionals outside of TRiP participated in Custom Coordinated Case Conferences? answer 8) What barriers to services and support has TRiP helped children, youth and family overcome? answer 9) Do you feel that TRiP has mobilized services around individuals? Please explain you answer. answer 10) Do you feel that TRiP has integrated services engaged with individuals? Please explain your answer. Answer 11) Do you have any other observations or feedback? Answer



TRiP Staff Closing Interview Guide 1) During your time with TRiP, what have you seen this initiative accomplish? 2) What has the collaboration among TRiP partners done for clients? 3) What has the collaboration among TRiP partners done for the participating agencies? 4) What are some of the successes you have experienced at TRiP? 5) What have been some of the challenges you have experienced? 6) What are some opportunities for improving TRiP? 7) How well have agencies and professionals outside of TRiP participated in Custom Coordinated Case Conferences? 8) What barriers to services and support has TRiP helped children, youth and families overcome? 9) Do you feel that TRiP has mobilized services around individuals? 10) Do you feel that TRiP has integrated services engaged with individuals? 11) What has been the key outcomes of TRiP on the clients you serve? 12) What do you feel are the key ingredients of the TRiP model? 13) What do you feel has been the overall impact of the reporting process on TRiP? 14) Do you have any other observations or feedback? 15) What has been the benefit for you as a professional to be part of TRiP?


TRiP Staff Initial Survey As part of the evaluation process, TRiP staff are asked to complete a confidential survey on their experience with TRiP to date. Your completion of this survey is completely voluntary. Choosing not to complete the survey will not impact your role in TRiP. Your answers will remain anonymous and confidential. The answers to the survey questions will only be seen and analysed by Dr. Chad Nilson. Results of the survey will be reported in the aggregate so that no sectors or individuals can be identified. Please send your completed surveys to Dr. Nilson at [email protected] by April 28th. Thanks so much. 1) What is your overall perspective on your experience at TRiP so far? answer 2) What has the collaboration among TRiP partners done for clients? answer 3) What has the collaboration among TRiP partners done for the participating agencies? answer 4) What are some of the early successes you have experienced at TRiP? answer 5) What have been some of the challenges you have experienced? answer 6) What are some opportunities for improving TRiP? answer 7) How well have agencies and professionals outside of TRiP participated in Custom Coordinated Case Conferences? answer 8) What barriers to services and support has TRiP helped children, youth and families overcome? answer 9) Do you feel that TRiP has mobilized services around individuals? Please explain you answer. Answer 10) Do you feel that TRiP has integrated services engaged with individuals? Please explain your answer. 11) What do you feel are the key ingredients of the TRiP model? Answer 12) Do you have any other observations or feedback? Answer


YOUTH EXIT SURVEY

To see how things are going, we want to ask youth to complete this survey. We will keep your

answers TOP SECRET—in fact we don’t even want your name. You don’t have to complete the

survey if you don’t want to. But it would be #REALLYAWESOME# if you did. Thank you so much.

PLEASE WRITE YOUR ANSWERS TO EACH QUESTION

Has anyOne hElped yOu latelY?

do you still need help?

IS THERE

ANYTHING

DIFFERENT IN

YOUR FAMILY?

(if so, what?)
