Survey Design:

Sharing What We’re Learning

Health Behavior Survey (HBS)

CASE

DPMDPNbS ODOS

VPSA

UVA

ESHC

OHP

Health Promotion Survey (HPS)

SOM

ISASGordie-CASE

Both Surveys

• Used for over 10 years• Stratified random samples of undergrads• Over-sampled first years & minority students• Web-based• Spring time• Alcohol use focus

Problems

Duplication of content, population, timing, uses Response rates Survey requests to students, especially of minority students Other surveys:

– NCHA every three years– eChug for first-year students: baseline & follow-up– smaller surveys related to celebratory drinking events:

• Halloween ● Last home football game• Spring break ● Foxfield

The Charge

Merge the HPS and HBS into a single survey

The Team

• Jennifer Bauerle

• Susie Bruce

• Adrienne Keller

• Jason Shaffer

Question Everything

• What is the primary purpose of this survey?• What do we need to know to accomplish that

purpose?• What kind of survey will best accomplish our

purpose?• How does each question contribute to

accomplishing the purpose?

Primary Purpose

To guide and evaluate social norms alcohol education interventions of OHP and CASE.

• Is there change?

• What changed?

• Who changed?

• Is change related to exposure to the interventions?

What We Need to Know

• Demographics • Drinking Behaviors and Attitudes• Perception of Drinking Behaviors and Attitudes• Exposure to Education• Outcome Measures

• Over time!

Over time!• Both HPS and HBS are REPEATED CROSS-SECTIONAL

– Annual– New sample each year– Can examine POPULATION change but not INDIVIDUAL change

• New Survey: LONGITUDINAL COHORT– Annual– Each participant receives the survey(s) for 4 years– Can examine INDIVIDUAL and POPULATION change

Enroll Graduate

What We Want• Longitudinal cohort study: Follow a random sample

of students through their undergraduate career at UVa

• Online• Multiple survey points each year:

– Yearly baseline: early Fall– Early December: survey focused on Halloween &

“4th year 5th”– Late spring: survey focused on Spring Break and

Foxfield

Population

Outcomes

Periodic follow-up for 4 years

Sample

Relevant characteristics

Exposure

Behaviors

Perceptions

Negative Consequences

Behaviors

Perceptions

Negative Consequences

InterventionProblems

?? Questioning Everything ??

How will we use the information?• Do we want to compare to national trends?

– Example: alcohol consumption

• Do we want to compare to institutional data?– Example: race/ethnicity

• What have we learned from past surveys?– Example: perception–reality gap

• How do we assess exposure?– Penetration and saturation measures

?? Questioning Everything ??

• What do we need for publication?– Example: Stages of change measurement

• What do we need for stakeholders?– Example: institution-specific events

• What answer options do we want?– Categories, Menu of choices, Open-ended

• How long will this take to complete?– “The one with the most data wins” vs– “The one with the smallest sample loses”

Design Process• Create a “map”• Collate existing survey questions into

the map• Frequent team meetings• Examine data from previous surveys• Review relevant literature• Cross-check with other surveys• Focus groups with audience members• Create online: SurveyMonkey or

institutional ITC• Revise, revise, revise

Our “Map”Purpose: Guide & Evaluate Social NormsAlcohol Educational Interventions of OHP/CASEIs there change?Who changed?Where is exposure the greatest?

Critical data for quant. analysis

(1)DEMOGRAPHICS1 Gender1 Ethnicity1 Year1 Age1 International1-4 Greek/Athlete1-4 Family history

(2)DRINKING BEHAVIORS & ATTITUDESAttitudesBehaviorsQuantityFrequencyBAC – weightCelebratory drinking

(3)PERCEPTION OF DRINKING B&A- Attitudes -- Acceptable- Behaviors -- Quantity-- Frequency

(4)EXPOSURE TO EDUCATION- Intervention frequency? Y/N?- List intervention -- SSJ -- G-wide -- ADAPT -- PHE -- AA…

(5)OUTCOME MEASURES- Protect. Behaviors- Neg. Consequences

Other comments…

?’s- Stages of change?- Most effective way to measure Q F- Reference Alcohol Wise & NCHA

Don’t - Involvement- Leadership- Drugs/Tobacco- Drinking since winter break- Where drinking happens

?- 2nd hand effects - Others drinking NCHA- Fake ID

Matching code INDIVIDUAL CHANGE OVER TIME

2,3

Some Social Norms Specific Issues

• Perception questions: – “Typical student” or “Most students”

• The “gap”: what do we need to know

• Wording for injunctive norms: – “I believe that…” – “I think that most students believe that…”

Getting to Go

• NIH Certificate of Confidentiality

• IRB approval

• VPSA approval

• Sampling frame

• Unexpected road blocks

Going, Going…GONE

• Timing• Online• Response Rates

– Incentives– Length of time it’s open– Reminders

2010 Survey Response Pattern

WEEK: 1 2 3 4 5

Questions, Comments, Differences?