Engaging stakeholders

What is the Centre?What is the Centre?

Knowledge

Bringing people and knowledge together to promote the best mental health and

well-being for every child and youth

Capacity Partnerships

Full continuum of effective and accessible mental health services for children and youth.

Webinar overviewWebinar overview This webinar will be recorded, so that it can be

available on the Centre’s website as an educational resource

The slides and any supplementary materials will be sent to participants at the end of the session

The session will take approximately 45 minutes including time for questions

You may ask questions or make comments at the end of the session by:• Typing your question or comment

Engaging stakeholdersEngaging stakeholders

Facilitators• Ilana Smyth, Research Associate, Centre of

Excellence• Paula Cloutier, Research Associate, CHEO

Who are the stakeholders of an evaluation and how do they influence evaluation? This webinar will discuss the importance of identifying stakeholders and how to engage them in planning and conducting the evaluation.

OutlineOutline

Stages in engaging stakeholders:• Identifying stakeholders and their needs• Techniques to identify stakeholders• Managing expectations and taking action

Shared experience from recent evaluation activities

Discussions and questions

Communication with StakeholdersCommunication with Stakeholders

Communicating with stakeholders is important throughout an evaluation

Intended users of the evaluation are more likely to use the results if they have been involved in the process

Identify stakeholdersIdentify stakeholders Stakeholders are those who can affect or are

affected by an evaluation process and/or its findings

• Individuals• Groups• Organizations

Stakeholders are individuals • Individuals within organizations • Individuals within groups

Identify stakeholdersIdentify stakeholders

Consider stakeholders who may have a negative influence on the evaluation• Including those who oppose using resources

for evaluation purposes• Including those who may feel threatened or

experience anxiety around certain aspects of the evaluation

Identify stakeholdersIdentify stakeholders

Keep the definition of stakeholders broad at first, however it’s difficult for an evaluation to answer all questions for all of those interested.

Necessary to then narrow down the evaluation questions and focus on a more manageable list of potential stakeholders• Primary intended users or key evaluation

stakeholders

TechniquesTechniques

List evaluation stakeholders• Brainstorm a list of individuals or groups

TechniquesTechniques Stakeholder influence diagram

Stakeholder:

Stake or Interest in the Program:• What do they want to get out of the program, and/or•What do they want the program to produce?

Stake or Interest in the Evaluation• What do they want to get out of the evaluation, and/or•What do they want the evaluation to produce?

TechniquesTechniques Power versus interest grid

Subjects – Have a significant

interest but little power

Players – Have a significant

interest and substantial power

Crowd – Have little interest and

not much power

Context Setters – Have substantial

power, but little direct interest

Power

Interest

Document stakeholder needsDocument stakeholder needs

Advance planning is important• Establish as early as possible the role and

function of stakeholders Establish ground rules and to what extent

stakeholders are involved in the evaluation• Establish timelines for involvement

Manage stakeholder expectationsManage stakeholder expectations

Stakeholder management – ways of working with positive as well as negative stakeholders

Better referred to as stakeholder engagement Begins in the early stages and continues

through development and implementation phases

Engaging stakeholders after the Engaging stakeholders after the evaluationevaluation

Tell your stakeholders how things went, both with the program and with the evaluation  

Follow up an evaluation report with a phone call or an in-person meeting, especially with primary stakeholders

Focus discussion on what the program has learned and how it intends to grow and improve, rather than just reporting on outcomes

Engaging stakeholders after the Engaging stakeholders after the evaluationevaluation

Invite stakeholders to brainstorm ideas on how to strengthen your program based on the evaluation findings

Send out a final evaluation report to all levels of stakeholders upon the program's completion

Specialized Psychiatric & Mental Health Outreach Specialized Psychiatric & Mental Health Outreach Services for Children and Youth in Eastern OntarioServices for Children and Youth in Eastern Ontario

SPMHS Outreach ServicesSPMHS Outreach Services

Services designed for children and youth who have complex mental health disorders which result in serious psychiatric, emotional and/or behavioural disturbance

Collaborative care approach

Role is to reinforce, support, provide consultation and training to community based MH services

Deliver services on a planned basis or by request either through travel to the community or by way of telehealth and telepsychiatry

Who are the Stakeholders?Who are the Stakeholders?

Funder: MCYS

Service Providers: CHEO (age 0-15), ROMHC (age 16-18)

-Clinicians (psychologist, psychiatrist, psychological associate, psychiatric nurse, occupational

therapist)

Rural Community Agencies: Providing MH services

Rural Physicians: referring

Consumers: Children, Youth, Caregivers

Discharge to Referring Physicians and/or Community

SPMHS Centralized IntakeAssess and refer to Appropriate Service/Program:

•Outreach•Outpatient•Inpatient•Emergency Department•Appropriate Community Services

ConsultationAssessmentClinical Care PlanTreatmentRecommendations

Renfrew

Prescott/Russell

SD&G

North Lanark

Ottawa

Referring Physicians

Not appropriate for SPMHS Services

Outreach Teams

North Grenville

Engaging the StakeholdersEngaging the Stakeholders

Consult with funders, service providers and consumers regarding outreach services (prior to program development)

Clarify needs, expectations and capacity on part of both specialized and community mental health care partners (prior to program development)

Develop services in response to identified local needs and local culture (be adaptive, enhance existing community services, address issues of role definition, responsibility….)

Evaluate outcomes leading to ongoing refinement and improvement of service (ideally build evaluation in right from the start)

Getting Service Providers On boardGetting Service Providers On board

Staff meeting(s) to discuss program and evaluation with plenty of time for Q & A

It is essential that those involved see the evaluation as meaningful and purposeful

Appropriate training be offered as necessary

Focus groups to get feedback from clinicians

Willingness to make changes along the way (evaluation is not research)

Getting Service Providers On board with the Getting Service Providers On board with the Evaluation: Success?Evaluation: Success?

County Accepted to

Outreach

Compliance with

evaluation

Stormont, Dundas & Glengarry

146 22 (15%)

North Lanark 132 8 (1%)

Renfrew 74 51 (69%)

Prescott-Russell 47 1 (2%)

North Grenville 19 12 (63%)

Ottawa 5 5 (100%)

Getting Clients On board with the Getting Clients On board with the Evaluation: Success?Evaluation: Success? Satisfaction Questionnaires (17% completion)

Consider more than one way to get feedback

(e.g., consumer focus groups)

Getting Agencies On board with the Getting Agencies On board with the Evaluation: Success?Evaluation: Success? Share evaluation outcomes with them (2/5 who

wanted to hear anything)

Keeping your Stakeholders EngagedKeeping your Stakeholders Engaged Communication is key

Continually demonstrate the relevance of the evaluation (one size doesn’t fit all)

Keeping the Evaluators EngagedKeeping the Evaluators Engaged

Don’t have an all or none attitude (view it as a process) Identify and celebrate the successes along the way (the smaller achievements add up) Learn from the failures (be willing to modify and try something

else) Plan your next steps (focus groups with referring physicians)

SummarySummary

Identify key stakeholders at the beginning of evaluation activities as much as possible

Engage stakeholders in evaluation activities from the beginning

Document stakeholder’s involvement Ongoing communication is key Flexibility is a necessity

For more informationFor more informationIlana Smyth

Research Associate, Centre of Excellence613.737.7600 Ext. 3325

ilsmyth@cheo.on.caPaula Cloutier

Research Associate, CHEO613.737.7600 Ext. 3672

cloutier@cheo.on.ca