Upload
jasmin-parker
View
215
Download
1
Tags:
Embed Size (px)
Citation preview
2006-07 System of Care Start-up Webinar Series
Making it Last…The Marriage of Social Marketing
and Evaluation
Social Marketing and Evaluation Webinar for the 2006 Newly Funded Communities
July 10, 2007
2006-07 System of Care Start-up Webinar Series
Social Marketers Are
From Mars
Evaluators Are From
Venus
Social Marketers Are Social Marketers Are From Mars…From Mars…
Like simplicityLike simplicity Comfortable with generalitiesComfortable with generalities Goal-orientedGoal-oriented Like to see quick resultsLike to see quick results Prefer short, straightforwardPrefer short, straightforward presentationspresentations Risk-takersRisk-takers
Evaluators Are From Evaluators Are From VenusVenus
Like complexityLike complexity Deal in absolutesDeal in absolutes Process-orientedProcess-oriented Like to take time, explore Like to take time, explore
all anglesall angles Prefer lengthy, abstract reportsPrefer lengthy, abstract reports Risk-averseRisk-averse
2006-07 System of Care Start-up Webinar Series
Mars and Venus Are Both Planets in the Same UniverseWant to help children: Stay in school Stay at home Become more productive members of
society at a cost that helps sustain programs in communities
2006-07 System of Care Start-up Webinar Series
Challenges and Barriers to a Happy, Productive RelationshipSocial Marketers Want interim
data to report Want easy to
communicate info
Have difficulty interpreting/ identifying alone
Evaluators Want to focus on
long-term outcomes
Want data communicated accurately
Uncomfortable passing along early results
2006-07 System of Care Start-up Webinar Series
Why Cooperate At All? Presentation of outcomes is key to sustainability Increase the public’s knowledge and awareness of an issue,
problem or solution Influence perceptions, beliefs and attitudes that may
change social norms Increase your organization’s visibility as a resource on
children’s mental health Enhance your organization’s leadership position Strengthen relationships Form a united front for projecting and reinforcing messages Create policy change Increase demand or support for community-based,
youth/family-driven, and culturally competent mental health services
Truly make a difference in the lives of children and families in your community
2006-07 System of Care Start-up Webinar Series
Why Cooperate At All? Evaluators have the information:
Descriptive/Outcome Data
Local Wraparound Fidelity Index
Service Experience Data
System of Care Assessment
Data Profile Reports
Continuous Quality Improvement Reports
2006-07 System of Care Start-up Webinar Series
Why Cooperate At All? Social marketers know how to present it:
Audience Research
Communication Strategies and Tactics
Packaging Data
2006-07 System of Care Start-up Webinar Series
Can This Marriage Be Saved?
2006-07 System of Care Start-up Webinar Series
Understanding Is the Key What are roles?
What is the language?
What are the desired outcomes?
2006-07 System of Care Start-up Webinar Series
Role of the Evaluator Understand program goals Analyze ongoing/post-program needs Design evaluation that captures pertinent
data Conduct evaluation; refine where
necessary Monitor other data sources; compare and
contrast Communicate regularly with staff across
program, particularly social marketers Report interim and final results Empower families and youth to be
persuasive advocates
2006-07 System of Care Start-up Webinar Series
Role of the Social Marketer Understand program goals Devise strategic communications plan Identify and profile key audiences Monitor changes and trends in attitudes
and behavior Share challenges and successes with
audiences Revisit and refine plan Communicate regularly with staff across
program, particularly evaluators
2006-07 System of Care Start-up Webinar Series
Role of Family and Youth Distinguish if the data reflects their
community Help social marketers package the data,
making it accessible and relevant Inform message development Complement data products with
descriptive testimonials Prevent dissemination of potentially
misleading data Provide the bridge from numbers to lives Promote retention and recruitment for
studies
2006-07 System of Care Start-up Webinar Series
What Can You Offer Each Other?Social Marketers Understanding of
audience needs Suggestions for
simpler language Clearer
communication of statistical info
Evaluators Accurate, timely
data on varied topics
Interpretation of results
Monitoring and analysis of data from outside sources
2006-07 System of Care Start-up Webinar Series
If We’re From Different Planets, Do We Need an Interpreter?
Social Marketers Audience
segmentation Editorial board Actuality
Evaluators Informed consent Program theory
model Longitudinal
statistical analysis
2006-07 System of Care Start-up Webinar Series
Breaking Down the Language Barrier Eliminate jargon where possible
Define terms
Ask questions
2006-07 System of Care Start-up Webinar Series
Making Beautiful Music Together
2006-07 System of Care Start-up Webinar Series
National Example Collaboration between the National
Evaluation Team and the Children’s Campaign has resulted in many products including: Fact sheets
PowerPoint presentations
Awareness Day data
2006-07 System of Care Start-up Webinar Series
Challenges To balance audience-friendly language
with accurate data
2006-07 System of Care Start-up Webinar Series
Lessons Learned Dialogue can start from either team
Begin working together at the very beginning of development
Collaborate on the development of a prototype
Create a timetable which includes time for multiple revisions
Learn about the other team's development process
Pose evaluation questions several different ways for better understanding
Keep the dialogue going!
2006-07 System of Care Start-up Webinar Series
Questions?
Please press *7 on your phone to unmute your line. When you are finished, please press
*6 to remute.
2006-07 System of Care Start-up Webinar Series
Local Example: Family Voices Network of Erie County We are in our 3rd Year – grant funded in
October 2004. We have an informal process for
communicating with evaluation– we’re located in the same building and maintain regular contact with the families. Sometimes this is a dual role between social marketing and evaluation staff.
We share data with our parents and youth.
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County: Social Marketing & Evaluation Team
Many times the social marketer and the evaluator wear dual hats.
We review data with our parents and youth and listen to their suggestions on how to make data family-friendly.
We inform the policy-makers of our findings based on the data collected bi-weekly at our Management Team meetings.
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County: Project Samples Community Assessment for Family
Involvement:
Knowledge & Effectiveness Survey
Children’s Mental Health Awareness Day:
NCMH Awareness Survey
Wraparound Bulletin:
Our quarterly newsletter
2006-07 System of Care Start-up Webinar Series
The 3 purposes of our Community Survey: The purpose of this survey is to identify community knowledge and attitudes
and to provide marketing assessment and direction including stigma evaluation.
The social marketing team presented the concept to the evaluation team and they worked together to develop a survey that would serve as a baseline of where our community stands today. By doing so, we have a measuring point to see improvements over time.
Between the evaluation team and the social marketing team, we identified our audience to be surveyed and worked together on identifying what we wanted to measure.
Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey
Category(as defined by survey code)
Total N sent N Returned Rate of Return
Care Coordinator 38 23 61%
Child Psychiatrist 2 1 50%
Family Medicine MD / DO 153 16 10%
Pediatrician MD / DO 152 29 19%
Judge 5 0 0%
Juvenile Justice 3 1 33%
Psychologist 81 18 22%
Supervisor CC 7 5 71%
Vendor 466 82 18%
Psychiatrist 19 1 5%
TOTAL 926 176 19%
Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey
We then rolled-up the data into four categories:
1. Families
2. Care Coordinators
3. Supervisors
4. Service Providers (vendor network)
2006-07 System of Care Start-up Webinar Series
We asked our families to tell us what was important to them from the survey categories. They identified:
1. Wraparound process/care coordination
2. The family role
3. Moving from system to natural supports
4. Making a referral
5. Child & Family Team (CFT) process
Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey
2006-07 System of Care Start-up Webinar Series
Next Steps:
• We will work with the evaluation team to create communications plans that address the needs of the family in the priority order they have identified.
• The evaluation team will provide us with other data from care management and other survey to assist in our plan development.
You can view the survey form on our Web site:http://www.familyvoicesnetwork.org//en/files/
Forms/Knowledge-Effectiveness-Survey.pdf
Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey
Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey
2006-07 System of Care Start-up Webinar Series
Local Example: Monterey County’s System of Care Mostly agricultural county, in many areas
60-75% Hispanic
Lead Agency: Monterey County Health Department, Children’s Behavioral Health (about 125 staff)
Social marketing and evaluation are led by outside consultants
To achieve optimal outcomes from social marketing & evaluation efforts both need to be recognized as integral parts of all other SOC activities.
And ensures that
programs have achievable outcome measures & a clear
identity & communication
plan.
Ensures that social marketing
activities have a built-in
evaluation component;
This allows the social
marketer to influence data collection from a creative and
audience perspective;
Program staff, social marketing, evaluation, families, youth & collaborating agencies work together from the very beginning – starting with the development of the social marketing plan, program logic models and design of outcome measures.
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:Strategy One: Bring Everyone to the Table
Family & Youth
Behavioral Health
Probation
Data Management
Cultural Competency
Social Marketing
EvaluationSocial Services
The Local Evaluation Team
Members of the Local
Evaluation Teambring
diversity & expertise
to the monthly and
“as needed” work sessions.
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:Strategy Two: Develop a Local Evaluation Framework
Monterey County utilizes a Comparison of Results Approach. Seeks to answer the questions:
How are we doing? Have outcomes for families and youth improved this year in comparison to last year?
13 local evaluation measures Developed local survey with
families and youth State and national data Program specific outcomes Data from partnering agencies
How do we bring partners together to discuss shared outcomes?
Monterey County’s System of Care:Strategy Three: Develop a “Content Matrix” to Share Evaluation Findings
Target Audience
Characteristics of Content
Information of Interest
Format
Families Bilingual, user-friendly. Personal stories and information. Avoid acronyms and terms that need further explaining (e.g., 200% poverty level).
Not interested in studies and data collection details. Data relevant to their child, e.g. “Last year in Monterey County, 80% of children diagnosed with a behavioral disturbance received a high school diploma.”
Fact sheet or newsletter with limited text, simple visuals, key facts without details, contact for more information.
Policy Makers, Dept. Directors
Limited time, prefer “bottom line” facts and visuals.
Outcomes, data sources, analysis “What does this mean to me and my constituents?” What is working, what isn’t?
Presentation, briefing sheet, Score Card.
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:Example: Bilingual Newsletter Reaching Families
Monterey County’s System of Care:Strategy Four: Develop a Distribution Plan and Implement it!
Target Audience
Format Distribution Who is responsible?
Families Newsletter Mailed to 350 families; 25 copies each dropped off at 8 clinic locations.
Family Partnership Program
Policy Maker, Dept Directors, Media, Public
Briefing
Score Card
Briefing scheduled for May 8th as part of Children’s Council meeting; Press release
Mailing (50 copies)
Project Director
Social Marketer
Staff Staff Memo
Presentation at all staff meeting.Article in staff newsletter
Staff mailboxes (135 copies) and email.Scheduled for March staff meeting.Staff mailboxes and email (article will go into February issue).
Admin
Project Director
Social Marketer
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:Example: Score Card
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:Score Card 2005- At a Glance
Purpose: To highlight system-wide (shared) accomplishments of Monterey County’s System of Care in a user-friendly way. To show why systems of care are important to our community and why it is important to continue to fund them.
Primary Target Audience: Policy-makers, Board of Supervisors, Children’s Council, Governance Council, Community Stakeholders and Staff (Language: Eng)
Distribution: Briefing to Children’s Council and media on Children’s Mental Health Summit Day; Mailing
2006-07 System of Care Start-up Webinar Series
Strategy One: Bring everyone to the table
Strategy Two: Develop a local evaluation framework
Strategy Three: Develop a content matrix to share evaluation findings
Strategy Four: Develop a distribution plan to share findings and implement it!
Strategy Five: Ask for feedback and actively involve new and existing partners in the process
Monterey County’s System of Care: Summary of Strategies
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:Lessons Learned & Accomplishments
Scorecard is a visually appealing and useful tool to explain SOC values and principles. Highlighting of shared programs and accomplishments generated goodwill – and momentum (many new partners this year!) Also- triggered more collaboration in other areas! Ongoing struggle – who is our target audience? Translate into Spanish? Weighing cost against level of distribution. Need to make the process youth- and family friendly – training. Goal in Monterey County: Bring more youth to the table.
Key lesson learned to new communities (from a social marketing perspective): Have the social marketer involved early on in the development stages of the evaluation framework and survey tool. The social marketer can provide input into how questions and outcomes can be translated into meaningful data to varies target audiences.
Making It LastMaking It Last Have early conversations about
goals/objectives Inform each other of
progress/challenges Be a resource for trends, new
developments, community news
Present information together for clarity and accuracy
2006-07 System of Care Start-up Webinar Series
Resources Caring for Every Child's Mental Health Campaign
www.mentalhealth.samhsa.gov/child/default.asp(Features an order form for Children's Campaign products such as Children’s Coloring Books, Annual Reports to Congress, etc.)
Subscribe to the listserve 4-THE-CHILD [email protected]
Family Leadership in Systems Evaluation Tip Sheets Engaging New Families in Evaluation & Language
Qualifying Family Preference in Hiring Federation of Families for Children’s Mental Health
The World of Evaluation: Three Short Courses
2006-07 System of Care Start-up Webinar Series
Questions?
Please press *7 on your phone to unmute your line. When you are finished, please press
*6 to remute.
2006-07 System of Care Start-up Webinar Series
Presenter email addresses: Deborah Porter,
[email protected] Lisa Rubenstein,
[email protected] Jana Sczersputowski,
[email protected] Tisha Tucker,