Upload
truongdan
View
215
Download
0
Embed Size (px)
Citation preview
Does Ethnically Stratified Address‐Based Sample Result in Both Ethnic and Class Diversity?
Case Studies in Oregon and Houston
Robyn Rapoport, Susan Sherr & David Dutwin
• Background for Research Question• Case Studies – Oregon and Houston• Glimpse into Future Research – Geographic Stratification by Income Level
Outline
Background for Research Question
• RDD coverage is declining, particularly of issue for smaller geography studies (such as state‐based studies) • The proportion of CPO households is rising
• Sampling techniques to reach cell phone residents with out‐of‐state area codes are not presently available
• ABS offers opportunity for full coverage – find CPO HHs and do micro‐targeting for sub‐populations
Literature Review
• Side‐by‐side studies using Address Based (AB) and RDD samples point to a bias in AB samples toward respondents who are non‐Hispanic whites and more educated• BRFSS Comparison: Link, et al (2006, 2008)• Massachusetts HIS: Sherr, et al (2009)
Case Studies
•• In 2010, SSRS conducted two large‐scale ABS studies using sample stratified by incidence of race/ethnicity
• Samples were stratified by census block group – more precise way to target specific geographic areas of ethnic minorities than phone exchange
• Primary goal of stratification plan – to increase the number of ethnic completes and meet geographic targets
Health of Houston Survey (HHS), on behalf of the
University of Texas, Institute for Health
Policy
Oregon Health Insurance Survey (OHIS) on behalf of Oregon Health
Authority’s Office of Health Policy and Research
Research question: To what extent does geographic targeting via ABS also help to address the ‘education gap’ since geographic areas of
greater ethnic diversity tend to be lower SES?
Oregon Health Insurance Survey• Sample Design:
• Extracted all HHs with Hispanic or Asian surname
• Flagged block groups with 10% or more AA, Asian, or Native American, or 30% or more Hispanic
• Residual block groups stratified in geographic strata to ensure minimum number of completes for each region
• 21 Strata:• 15 Geographic Regions• Ethnic Strata: High AA, High Hispanic,
High NA, Hispanic Surname, Asian Surname
Health of Houston Survey• Sample Design:
• Sample first stratified by the 7 SuperPUMAs in Harris County
• Extracted all HHs with Vietnamese or other Asian surname
• Flagged block groups with 50% or more AA, 50% or more Hispanic, 10% or more Vietnamese, and 15% or more Asian
• 42 Strata• Within 7 SuperPUMAs, sample
stratified by Ethnicity: High AA, High Hispanic, High Vietnamese, Asian Surname, Vietnamese Surname, Residual Strata
Stratification Procedures
General Specifications
3 Modes• Telephone• Web• Hard copy
CATI and web offered in Spanish
3 Modes• Telephone• Web• Hard copy
All modes offered in Spanish and Vietnamese
Gaining CooperationMatched Sample Unmatched SampleAddress matched w/landline number
Unlisted, CPO, or no phone
Advance letter* ● ●
Advance letter mentions post incentive ●
Post card reminder ● ●
Up to 20 call attempts ●
Refusal conversion via telephone ●
Reminder letter with hard copy q’re** ● ●
*$2 pre‐incentive with Houston advance letters**Letter and instrument were sent a second time to Oregon non‐responders
Stratification by ethnic groups achieved primary goal of increasing ethnic completes
Ethnic/Race Completes (n=10,143)
Ethnic/Race Completes (n=5,082)
weightedto SRS*
unweightedcompletesl
White 87.9% 85.6%Black 1.7 2.0Asian 2.6 3.4NA 2.5 3.0Hispanic 4.5 5.2Other .8 .9
weighted to SRS*
unweightedcompletes
White 48.4% 40.5%Black 18.8 19.0Asian 4.8 13.8Hispanic 25.2 24.0Other 2 2
*Simulated ‘de‐stratification’ using Simple Random Sample (SRS) baseweight
More non‐response among lower SES; unmatched sample is more educated than matched
7%
26%
35% 33%
5%
20%
38% 37%
LT HS HS SomeCollege
College +
Denotes significance at the 0.05 levelOregon Matched n=4,829; Unmatched: n=3,930Houston Matched n=3,275; Unmatched: n=1,784
11%
22%26%
41%
5%
18%
33%
44%
LT HS HS SomeCollege
College +
12% 25% 37% 27%ACS 21% 28% 26% 25%
8%
31% 33%28%
11%
34% 34%
21%
4%
16%
38%42%
6%
26%
37%31%
LT HS HS Some College College +
Education by mode
Denotes significance at the 0.05 level, but not all significant differences are highlighted on this slide.
(n=2,505) (n=4,346) (n=1,026)
Higher educational attainment associated with web and hard copy modes
(n=882)
20%
26%23%
31%
3%
13%
27%
57%
1%
22%
41%36%
LT HS HS Some College College +
Denotes significance at the 0.05 level, but not all significant differences are highlighted on this slide.
Education by mode
(n=2,082) (n=1,870) (n=1,107)
Starkest differences noted between WEB and CATI
HYPOTHESISStratification by income at the block group level may help address bias since lower
income levels correlate modestly with lower educational attainment
Created variable to estimate sample demographics if stratified block groups by average income level
Glimpse into the future……income stratified by geography
Income Stratification Procedure
Low income50%
Middle income30%
High income20%
• MSG provided income by census block group• Divided census block groups into thirds: high, medium, and low income• Created variable in dataset • Weighted income groups to approximate income‐based stratification
8%
28%
35%
29%
6%
22%
39%34%
LT HS HS SomeCollege
College +
Income‐stratified
6%
26%
35% 34%
5%
20%
39% 37%
LT HS HS SomeCollege
College +
Weighted to SRS*
Income stratification reduces overall educational levels but does not eliminate the gap by sample type
Matched n=4,829; Unmatched: n= 3,930
*Simulated ‘de‐stratification’ using Simple Random Sample (SRS) baseweight
16%
25% 26%
33%
8%
20%
35%37%
LT HS HS SomeCollege
College +
Matched n=3,275; Unmatched: n=1,784
11%
22%26%
41%
5%
18%
34%
42%
LT HS HS SomeCollege
College +
Income‐stratified Weighted to SRS*
*Simulated ‘de‐stratification’ using Simple Random Sample (SRS) baseweight
Summary
• ABS offers potential for best sample coverage of small geographic areas
• Stratification by ethnic group can increase the number of ethnic completes vs. what would have been attained otherwise; however, reaching unmatched households with lower SES remains problematic – this population is harder‐to‐reach and less likely to complete either the hard copy or online versions of the survey.
• Stratification by income may be a somewhat more effective way of increasing the proportion of completes with lower SES HHs…but disproportionate non‐response is likely to be an issue with alternate stratification plans also.
Implications
• May want to experiment with…• Larger incentives offered to block groups likely to be of lower SES to encourage respondents to call in or complete the hard copy or online instrument
• Incenting unmatched HHs to provide telephone numbers so that outgoing calls can be made
Thank You
Robyn [email protected] West Baltimore PikeMedia, Pennsylvania 19063www.ssrs.com@ssrs_solutions