View
216
Download
0
Tags:
Embed Size (px)
Citation preview
1
MSU Model Grant Overview
Dennis Martell
Sandi Smith
Michigan State University
MSU MODEL GRANT
• What Makes MSU a Model Program?• 1. Unique make up of team
– Service, Survey, and Academic Experts• 2. Effectiveness of global campaign to
reduce extreme drinking.• 3. Unique application of social norms
messaging on celebratory events.• 4. Unique focus on use of protective
behaviors to reduce harm.
3
Outline of the Presentation
• Overview of evaluations• Intervention design and evaluation• Social norming models• MSU’s social norming program and
evaluation design• Formative research on celebratory
drinking• Dissemination and enhancement
strategies
4
Evaluations in the Life Course of Programs
Adapted from: Rossi, P., and H. Freeman, Evaluation: A Systematic Approach
OutputOutcome/ImpactCost/BenefitCost Effectiveness
Decision to Act
Needs AssessmentFormative Evaluation
ProcessConformity to PlanTarget Reach/Coverage
Program Life Course
Implementation
Institutionalization
Assessmentof Utility
Outcome
Planning Monitoring
Problem EvaluationsConceptualization & Design
5
To Evaluate a Program's Outcome, We Need to Know . . .
What was the program supposed to accomplish? (Problem to change) How was it to accomplish this? For whom?
Why was the intervention supposed to have the intended effect? (Conceptual Model)
Did change occur? (Outcome/Impact) How much? Is the change associated with the program elements? Can alternative explanations for the change be ruled out? How much did it cost?
6
The Social Norming Conceptual Model of
High-Risk Drinking
Harm
Consumption
Perception ofNorms:
Descriptive
Perception ofNorms:
Injunctive
FamiliarityWith theCulture
DesireTo be
Normal
Alcohol poisoningPhysical injuryAuto accidentsViolenceSexual assaultSexual risk behaviorAcademic performance
TypeQuantity AbsorptionDurationPace
ProtectiveBehaviors
Setting limits 1 drink per hourDesignated driver Drink look-a-likeWatch out for friends Stay with same groupStay with same alcohol Eating food
7
The MSU Social Norming Intervention Strategy for High-Risk Drinking
Harm
Consumption
Perception ofNorms:
Descriptive
Perception ofNorms:
Injunctive
FamiliarityWith theCulture
DesireTo be
Normal
ProtectiveBehaviors
Intervention:Posters
Table Tents
Intervention: Flyers
Newspaper Ads
AOP
8
The Initial Question:
Was there a discrepancy between the perceived norm around drinking and the actual drinking norm?
i.e., Was a Social Norming approach an appropriate intervention strategy?
9
Difference Between Perceived Norm and Actual Behavior Before Starting Social Norming Campaign
6.1
5.4
Perceived Norm(Mean)
Actual Norm (Mean)
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
Number of DrinksLast Time Partied/Socialized
2000
Source: MSU NCHA 2000
The average perceived number
of drinks consumed by the typical student the
last time s/he partied is 13%
greater than the actual average
}
Difference Between Perceived Norm and Actual Behavior Before Starting Social
Norming Campaign
10
Overall Actual %
0-4 = (46.6%)
% Who Believe Typical Student Drank 0-4 Drinks Last Time Partied/Socialized by Gender and Status, 2000
Grad % 0-4 = 57.8%
Undergrad % 0-4 = 43.7%
Source: MSU NCHA 2000
2007 National Social Norms Conference 11
Examples of Ads, etc.
12
Data Collection Strategy for Evaluation
• Formative – Focus groups for ads
• Monitoring/Process – Web survey near end of each semester– Undergrads only– N = 1,000 – 1,200 @– Perceptions of others, celebration event drinking,
protective behaviors, familiarity with ads• Monitoring/Outcome
– NCHA (web) every 2 years (Feb.-March)– Grad and Undergrad– N = 900 – 1,400– Overall drinking, harm, protective behaviors
13
The Evaluation Questions
• Did the interventions reach the targeted audience?
• Did change occur in normative perceptions? • Is change associated with the intervention? • Did change occur in protective behaviors? • Did change occur in drinking behavior?• Did change occur in amount of harm?• Is there reason to believe this isn’t just
coincidence?
14
Did the Interventions Reach the Targeted Audience?
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
• Males as likely to report seeing ads as females
• Freshmen more than Sophomores more than Juniors more than Seniors
• Average number times reported seeing ads varied from 2.8 (Fall, 06) to 7.5 (Spring, 04)
• No difference between males and females
• Generally, Freshmen saw more times than Sophomores more times than Juniors more times than Seniors
Source: Celebration Surveys 2-9, n=1,211; n=1,040; n=1,277; n=1,073; n=1,334; n=1,110; n=891; n=1,405
% of Undergrads Reporting Seeing at Least 1 of the Norming Ads During the Semester, by
Semester and Year
15
Are Respondent Reports of Seeing Ads Reliable?
22.826.4
42.3
51.3 52.3
34.2
41.2
3.3 3.4 3.30
10
20
30
40
50
60
Ad 1 Ad 2 Ad 3 Ad 4 Ad 5 Ad 6 Ad 7 FakeAd 1
FakeAd 2
FakeAd 3
PERCENT
Source: Spring Celebration Survey, 2005
% of Respondents Who Claimed to Have Seen Each Ad/Poster
2007 National Social Norms Conference 16
Percent
Students’ Latitude of Acceptance/Rejection of Ad Claims
Ave. Number Drinks R’s Believe Typical Student Drank
“For each percentage, please indicate if you find the statement to be very believable, somewhat believable, or not believable at all.”
"The percentage of MSU students that typically drink 4 or fewer drinks when they party is ___% "
Source: Spring Celebration Survey, 2005, n=1,073
Actual % of Respondents who drank 0-4 Last Time
17
Did Change Occur in Normative Perceptions?
Average Number of Drinks the Typical Student Believed to Have Consumed the Last Time Partied/Socialized
Source: MSU NCHA 2000-2006
}A 19%
Decrease
}
A 6.8% Decrease
5.485.63
4.644.50
6.24 6.36
5.93 5.87
2000 2002 2004 20063.00
4.00
5.00
6.00
7.00
Undergrads Grads
Did Change Occur in Normative Perceptions?
18
Percent
% of Respondents Who Believe the Typical Student Drank 0-4 Last Time Partied, by Gender and Student Status
Source: MSU NCHA 2000 - 2006
2007 National Social Norms Conference 19
Did Change Occur in Use of Protective Behaviors?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
2000 2002 2004 2006YEAR
% W
ho A
lway
s or
Usu
ally
a. Alternate drinks
b. Plan not to exceedlimitc. Choose not to drink*
d. Have designateddriver*e. Eat before or during*
f. Have friend saywhen had enough*g. Keep track of drinks
h. Pace drinking
i. Drink look alikes
% of Undergrads Who Always or Usually Use Various Protective Behaviors, by Year
20
Change in Use of Protective Behaviors
30.0
35.0
40.0
45.0
50.0
55.0
60.0
65.0
70.0
2000 2002 2004 2006
Did none
Did 1 or more
Among those who drank 5+ drinks last time partied
2007 National Social Norms Conference 21
22.624.4
20.222.1
11.7
17.4 17.616.1
65.7
58.3
62.2 61.8
10
15
20
25
30
35
40
45
50
55
60
65
70
2000 2002 2004 2006
Never/Not in past 30 1-2 days 3+ days
Did Change Occur in Drinking Behavior?
PercentOf All
Students
Number of Days in Previous Month Drank Alcohol
5.9% decrease from 2000 in percentage of students drinking 3 or more days per month
22
Did Change Occur in Drinking Behavior?
5.42
4.98 4.90 4.97
5.63
5.38 5.32 5.35
4.64
3.313.17 3.093.00
3.50
4.00
4.50
5.00
5.50
6.00
2000 2002 2004 2006
Year
Mea
n #
Dri
nks
All Students
Undergrads
Grad & Other
Mean Number of Drinks Last Time Partied/Socialized, by Year
23
Is Drinking Related to the Likelihood of Harm?
• Physical/Psychological– Injury to self as result of alcohol– Injury to other as result of alcohol– Involved in fights– Did something later regretted – Forgot where you were/what did – Had unprotected sex– Had forced sex
• Academic
– Lower grade on exam or project– Lower grade in course– Incomplete or dropped course
Types of Harm Asked About in NCHA
24
Is Drinking Related to the Likelihood of Physical/Psychological Harm?
Risk of At Least One Undesirable Event by Number Drinks Consumed Last Time Partied
0%10%20%30%40%50%60%70%80%90%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Number Drinks Last Time Partied
% H
ad A
t Le
ast
One
Und
esira
ble
Eve
nt in
Las
t
Sch
ool Y
ear
25
% of Students Experiencing Harm by Drinking Levels
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
HeavyModerateLight
Odds Heavy vs. Mod 2.4 4.0 2.8 1.7 1.9 1.5 2.2 1.6
Odds Mod vs. Light 18.4 43.2 33.7 7.9 12.4 12.5 5.7 7.4
Odds Heavy vs. Light 43.4 171.0 94.8 13.2 24.1 18.5 12.6 11.8
Heavy = Drink 10 or more days per month and 5+ drinks last time partiedLight = Drink 2 or fewer days per month and 2 or fewer drinks last time partiedModerate = Everything else
Source: MSU NCHA 2000, 2002, 2004, 2006; n = 4,155
2007 National Social Norms Conference 26
Did Change Occur in Amount of Harm?
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
2000 2002 2004 2006
Year
% o
f R
esp
ond
ents
Did something later regretted*
Forgot where you were/what did
Injury to self as result of alcohol
Had unprotected sex*
Involved in fights
Injury to other as result ofalcoholForced sex*
Source: MSU NCHA 2000-2006
Among those who drink This school year, as a consequence of your drinking, have you experienced. . .
8.5% reduction from 2000 to 2006
34.8% reduction from 2000 to 200617.0% reduction from 2000
to 200678.3% reduction from 2000 to 2006
27
Change in Academic Harm, by Year
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
2000 2002 2004 2006
YEAR
Per
cen
t
Alcohol effectedAcademicPerformance
Lower gradeexam or project
Lower grade incourse
Incomplete orDropped Course
28
Summary
Evidence of initial misperception of drinking norm Evidence norming campaign reached targeted
audience Evidence perception of norm changed Evidence use of protective behaviors changed Evidence drinking behaviors changed Evidence for reduction of drinking related harm
• Program is working
29
Is There Reason to Believe This Isn’t Just Coincidence?
• Dejong et al., 2006. “A multisite randomized trial of social norms marketing campaigns to reduce college student drinking” Journal of Alcohol Studies 67: 868-879.
• From 2000 to 2003, found experimental group of colleges implementing social norming campaign had no or very small increases in perceptions of drinking norms, actual consumption and harm, while control group of colleges and universities not implementing social norming campaigns had substantial increases in all three.
• The magnitude of the effect at experimental sites correlated positively with the intensity of the norming campaign.
• The secular trend in the time period was for drinking and harm to increase.
• Evidence MSU’s results are not just coincidence because of a secular trend.
Respondents’ Assessments of Social Norming Ads Distributed in 2003-05
Ad % Who
Saw
Mean #
Times
Saw
% Saw as
Believable
% Saw as New
Information
Tailgating ad 40.7% 2.7 42.4% 60.5%
Designated driver ad 36.6% 2.9 73.9% 41.8%
Basketball playing ad 27.3% 2.6 52.9% 68.7%
Pool playing ad (Fall) 36.9% 2.6 45.3% 56.0%
Moderation ad (Fall) 41.2% 2.9 59.6% 58.1%
Pool playing ad (Spring) 61.8% 3.1 39.3% 54.4%
Moderation ad (Spring) 52.3% 3.1 54.6% 54.3%
Spring Break: snow 18.8% 2.3 59.4% 41.1%
Spring Break: luggage 14.8% 2.3 62.8% 41.0%
MSU basketball 26.3% 2.4 55.9% 44.2%
Respondents’ Assessments of Social Norming Ads Distributed in 2003-05
Ad/Poster % Who
Saw
Mean #
Times
Saw
% Saw as
Believable
% Saw as New
Information
Sidelines ad (football) 31.8% 2.6 61.1% 50.0%
Spartans Think (fall) 48.0% 2.5 60.2% 63.3%
Spartans Do (fall) 50.5% 2.5 68.7% 68.4%
Spartans Do (spring) 53.8% 2.8 76.0% 61.4%
Spartans Think (spring) 52.0% 2.5 59.7% 56.7%
St. Patrick’s ad (spring) 46.4% 3.0 62.3% 55.2%
Courtside ad (spring) 32.6% 2.6 68.8% 43.4%
Courtside ad (Izzone) 44.9% 2.1 57.4% 40.3%
Halloween ad 25.4% 2.0 68.5% 39.9%
Halloween Poster 29.3% 3.5 73.6% 52.4%
Respondents’ Assessments of Social Norming Ads Distributed in 2006-7
Ad/Poster % Who
Saw
Mean #
Times
Saw
% Saw as
Believable
% Saw as New
Information
Global ad (Spring) 25.5% 2.5 77.7% 29.0%
Courtside ad 18.0% 2.1 78.2% 37.8%
Izzo ad 38.5% 2.8 79.6% 51.8%
Izzo Poster 38.1% 3.1 79.0% 47.o%
Spring Break Ad 42.6% 2.5 72.0% 52.0%
Spring Break Table tent 48.2% ** 73.4% 56.9%
Global Scrambled (Fall) 43.1% 2.9 68.8% 44.3%
Spartan Do TT 59.6% ** 60.7% 70.8%
Halloween TT 44.4% ** 72.3% 47.9%
Halloween ad 48.3% 2.1 79.2% 50.5%
*Ad Believability and Over and Under Estimation of Drinking Norms
This study looked at estimation and accuracy of normative perceptions for students during
both everyday drinking and celebration drinking.
Believability and Estimation
• It found that students who drank less than four
drinks underestimated the norm, and those who drank more than five drinks overestimated the norm.
• Ad believability played a crucial role in this
process. Those who believed the ad more closely estimated alcohol consumption by their peers.
Effectiveness
• The reduction of specific primary harms (NCHA and Celebratory Survey data)
• The increased adoption of protective behaviors across the target population
• The level of acceptance the campaign has received from students, including levels of message believability and message usefulness
Ethnographic and Formative Research Findings
“Celebratory Occasions”
U.S. Department of Education Grant 2001-2003
DRINKING AND DRUNKENNESS
ON CELEBRATORY OCCASIONS
Holiday DRINK DRUNK
Occasion: PREVAL RATE
Halloween 32% (57%)
St. Patrick’s 26% (58%)
Typical Thursday 19% (48%)
DRINK PREVAL = Percent of all students that report drinking
DRUNK RATE = Proportion of drinkers who report getting drunk
DRINKING AND DRUNKENNESS
ON CELEBRATORY OCCASIONS
Football DRINK DRUNK
Occasion: PREVAL RATE
Big Gameday 38% (56%)
Other Games 37% (50%)
Typical Saturday 23% (39%)
DRINK PREVAL = Percent of all students that report drinking
DRUNK RATE = Proportion of drinkers who report getting drunk
DRINKING AND DRUNKENNESS
ON CELEBRATORY OCCASIONS
Weeklong DRINK DRUNK
Occasion: PREVAL RATE
Spring Break 48% (62%)
Welcome Week 37% (72%)
End of Semester 23% (55%)
Typical Week 47% (48%)
Figure 4. Percentage Drinking Less or More than Planned
0
5
10
15
20
25
30
Events
Per
cent
age
Less
More
Non-Drinking Norm Prevalence:
76% End of Semester
74% St. Patrick’s
68% Halloween
62% Big Game
60% Welcome Week
52% Spring Break
Non-Drunkenness Norm Prevalence:
(Moderately or not at all)
86% End of Semester
85% St. Patrick’s
82% Halloween
79% Big Game
71% Welcome Week
70% Spring Break
Types of Drinkers:
“Anytime” Drinkers -- 54%
Drink on typical days and special days
“Celebration” Drinkers -- 35%
Drink on special days but not typical days
Types of Drinkers:
“Non-Celebration” Drinkers -- 2%
Drink on typical days but not special days
Seldom Drinkers -- 9%
Drinkers who didn’t drink on the special or
typical days measured in survey
Table 5. Percentage Distribution of Drinking Types, by Demographic Characteristics
n Anytime Celebration Seldom Non-Celebration ……
Gender Female Male
514
431
49.8%
58.9%
35.0%
34.1%
12.5%
4.9%
2.7%
2.1%
19.03
p<.001
Status Freshman Sophomore Junior Senior
180
212
242
313
47.2%
54.7%
55.0%
56.5%
39.4%
35.8%
33.1%
31.9%
11.7%
6.6%
8.7%
9.6%
1.7%
2.8%
3.3%
1.9%
8.66
N.S.
Race White African American Other
793
61
91
56.6%
32.8%
45.1%
33.7%
37.7%
40.7%
7.9%
26.2%
6.6%
1.8%
3.3%
7.7%
42.45
p<.001
Drank in YesH.S. No
666280
57.8%44.6%
34.1%36.1%
6.0%16.4%
2.1%2.9%
30.6p<.001
“Focused” Findings
• Identified Protective Behaviors showed a reduction in harm independent of consumption
• Normative climate seemed to approve of drinking and higher than normal consumption on celebratory occasions
• Individuals identified as ‘celebratory’/event specific drinkers!
Protective Behaviors:
Promising protective behaviors not necessarily related to consumption: (Celebration ‘02)
– Staying with same group of friends-
– Remaining in one location-
– Consuming only one type of alcohol.
Findings from the Social Norming
Study of Student Alcohol Use2003-05
Student approval of activities
Respondents were asked about their approval of various activities, and also their estimates as to whether ‘most MSU students’ approve the same behavior.
• The following slides show the results as a function of level of student drinking
Perception of various celebrations: Celebration Survey data
0%
10%
20%
30%
40%
50%
60%
70%
80%
WelcomeWeek
Tailgate Halloween St. Patrick'sDay
SpringBreak
EstimateActual
Approval & estimates of celebration drinking
76%69%
24%
68%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Estimate: MSU Celebrationdrinking
Yes
No
Having more than 5 drinks
-100%-80%-60%-40%-20%
0%20%40%60%80%
100%
OVERALL LEVEL L LEVEL M LEVEL H
Dis
appr
ove
App
rove
I approve Most approve
Having more than one drink/hour
-100%-80%-60%-40%-20%
0%20%40%60%80%
100%
OVERALL LEVEL L LEVEL M LEVEL H
Dis
appr
ove
App
rove
I approve Most approve
Getting wasted
-100%-80%-60%-40%-20%
0%20%40%60%80%
100%
OVERALL LEVEL L LEVEL M LEVEL H
Dis
appr
ove
App
rove
I Most
Celebration drinking
-100%-80%-60%-40%-20%
0%20%40%60%80%
100%
OVERALL LEVEL L LEVEL M LEVEL H
Dis
appr
ove
App
rove
I approve Most approve
Results of Note: Injunctive Norms
• 77% of MSU students approve of drinking more than usual during celebration events.
• 74% of MSU students approve of getting drunk during welcome week.
• 73% of MSU students report they approve of getting drunk during Halloween.
Celebrations Surveys 2003-2006
Dissemination Strategies for the Model Grant
• Develop and evaluate a website that will provide interactive instructions for researchers and practitioners wanting to replicate our program
• Submit content to Wikipedia.org on social norms• Create and evaluate a handbook on the
development and operationalization of a social norms marketing campaign
• Provide service as consultants to other institutions of higher education via hosting several teleconferences/webinars
• Present our model at workshops and conferences
Enhancements
• Enhancements to the program include:– the development of messages that shift
toward subjective norm use distributed through social-interactive community sites (Facebook)
– the use of the Audience Response System—(ARS) or "clicker technology" to enhance live presentations