Embed Size (px)
Citation preview
1
Renee M. GindiNCHS
Federal Conference on Statistical MethodologyStatistical Policy Seminar
December 4, 2012
Responsive Design on the National Health Interview Survey: Opportunities
and Challenges
Division of Health Interview Statistics
National Center for Health Statistics
2
Objectives
National Health Interview Survey (NHIS) background
Potential features of responsive design on NHIS
Opportunities
Challenges
3
Conducted by National Center for Health Statistics
Nationally representative Representative monthly sample
In-person interviews 35-40,000 household
interviews/year
Fielded by U.S. Census Bureau ~700 interviewers in 6 regional
offices
The National Health Interview Survey (NHIS)
1 hour face-to-face interview – no incentives
4
1970
1975
1980
1985
1990
1995
2000
2005
2010
0%
10%
20%
1.3%
12.9%
Refusal Rates, NHIS 1969-2011
5
Sources of Paradata on NHIS
Contact History Instrument (CHI)
Used on other surveys fielded by Census
Front/Back sections of the survey instrument
Tailored to NHIS
Blaise audit trails
Used to produce item/interview times
6
Recent Paradata Research from NHIS
Using Statistical Process Control to monitor data quality estimates (item nonresponse, item time) over time
Using CHI variables to estimate response propensity response propensity and measurement error response propensity and survey outcomes
7
Looking Ahead: Responsive Design on NHIS
Some elements of responsive design Monitoring performance indicators Change design based on monitoring survey outcomes Target interventions to subsets using response
propensity
Timeline: 2016 sample redesign
8
Looking Ahead: Responsive Design on NHIS
Opportunities Real-time access to operations data New ways to estimate survey quality
Challenges Selecting and prioritizing survey outcome estimates How, when, and where data collection phases
should shift
9
Real-Time Access to Operations Data
Census Bureau’s Unified Tracking System (UTS) More information to make better decisions quickly Daily data update and historical data Flexibility in reports
NHIS-specific indicators on tracked on UTSDemographicRaceIncomeEducationEmployment
HealthUsual place of careNeeds help with personal
care Response qualityFirst /Last NameConsent for linkageAdult SSNTelephone number
10
New Ways to Estimate Survey Quality
Trying to identify measures that can help assess, reduce, and correct for nonresponse bias in our health estimates
Adding new interviewer observation questions on responders and non-responders Physical condition of the sample unit Household income, employment status Health-related indicators
11
Identifying priority estimates: 15 Selected Health Measures
Lack of health insurance coverage and type of coverage
Usual place to go for medical care
Obtaining needed medical care
Receipt of influenza vaccination
Receipt of pneumococcal vaccination
Obesity Leisure-time physical
activity Current smoking Alcohol consumption Human immunodeficiency
virus (HIV) testing General health status Personal care needs Serious psychological
distress Diagnosed diabetes Asthma episodes and
current asthma
12
“Phase Shifts”: How, When, and Where?
How can we “sufficiently alter” NHIS protocol? Mode shift? Shift to “core” survey? Introduce incentives?
When should protocol be altered given a monthly sample and production cycle? Is a 7-10 day window wide enough to achieve response
goals?
Where should protocol be altered? Nationally? Regional Office? State?
14
EXTRA SLIDES
15
SPC: Sample Adult Interview Pace (seconds per question) Control Chart, Regional Office 1,
Cluster 4
Z=8.1
UCL
LCL
13
4
6
8
10
12
S
UCL
LCLJan
2008
Mar
2008
May
2008
Jul
2008
Sep
2008
Nov
2008
Jan
2009
Mar
2009
May
2009
Jul
2009
Sep
2009
Nov
2009
Jan
2010
Mar
2010
May
2010
Jul
2010
Sep
2010
16
Response Propensity and Measurement Bias
LOW MEDIUM HIGH-2.50%
-2.00%
-1.50%
-1.00%
-0.50%
0.00%
0.50%
1.00%
1.50%
Social Security, Supplemental Security Income, Medicare
RESPONSE PROPENSITY
Bias
(Sur
vey
mea
n - R
ecor
d m
ean)
SS
SSI
Medicare
17
Correlations between CHI Measures and Participation and Health Outcomes : NHIS, 2010
CHI Measure
Absolute Value of Correlation with Family/Sample
Adult Participation
Absolute Value of Correlations with
76 Health Outcomes
Range Average
Refusal concerns .73/.50 .00 - .10 .04
Time constraints .15/.42 .00 - .14 .05
Privacy/trust .45/.38 .00 - .11 .04
Gate keeping .27/.32 .00 - .11 .03
Number of concerns .62/.45 .00 - .09 .03
Number of contact attempts .49/.31 .00 - .25 .07
Health problem .19/.18 .01 - .33 .13< .30: Weak/very weak .30 - .69: Moderate >= .70: Strong/very strong
