Upload
javion-bonde
View
219
Download
6
Tags:
Embed Size (px)
Citation preview
An Assessment of the Impact of Two Distinct An Assessment of the Impact of Two Distinct Survey Design Modifications on Health Survey Design Modifications on Health
Insurance Coverage Estimates in a National Insurance Coverage Estimates in a National
Health Care SurveyHealth Care Survey
Steven B. Cohen, Trena Ezzati-Rice, and Marc ZodetSteven B. Cohen, Trena Ezzati-Rice, and Marc Zodet
International Total Survey Error Workshop 2010 Stowe, VT
June 13-16, 2010
Background and motivation for Background and motivation for research research
The Medical Expenditure Panel Survey The Medical Expenditure Panel Survey (MEPS) is a national resource to inform health (MEPS) is a national resource to inform health care policy care policy
MEPS is a key survey resource to monitor:MEPS is a key survey resource to monitor:– Trends in estimates of the uninsuredTrends in estimates of the uninsured– Population characteristics of the uninsured Population characteristics of the uninsured – Duration of spells of uninsurance and long term Duration of spells of uninsurance and long term
uninsureduninsured– Financial consequences of being uninsuredFinancial consequences of being uninsured– Relationship between uninsurance and health Relationship between uninsurance and health
statusstatus
Thus need for timely, high quality MEPS dataThus need for timely, high quality MEPS data
MEPS survey backgroundMEPS survey background
Annual survey since 1996; nationally representative sample of Annual survey since 1996; nationally representative sample of householdshouseholds
5 rounds of data collection covering 2 calendar years5 rounds of data collection covering 2 calendar years
Used to estimate medical care utilization, access to care, and Used to estimate medical care utilization, access to care, and health care expenses for the U.S. civilian noninstitutionalized health care expenses for the U.S. civilian noninstitutionalized populationpopulation
Integrated survey designIntegrated survey design
– Each annual sample is a subsample of responding households (from Each annual sample is a subsample of responding households (from prior year) from another large ongoing U.S. health survey, the National prior year) from another large ongoing U.S. health survey, the National Health Interview Survey (NHIS)Health Interview Survey (NHIS)
Overlapping panel designOverlapping panel design
– Data from 1st year of new panel combined with data from 2nd year of Data from 1st year of new panel combined with data from 2nd year of previous panelprevious panel
Illustration of how panels/rounds Illustration of how panels/rounds comprise MEPS calendar year datacomprise MEPS calendar year data
MEPS survey design modifications in MEPS survey design modifications in 2007: Panel 122007: Panel 12
1.1. Re-engineered CAPI survey instrument: Windows-based platform Re-engineered CAPI survey instrument: Windows-based platform replaced the DOS-based system (questionnaire remained virtually replaced the DOS-based system (questionnaire remained virtually unchanged)unchanged)
2.2. New sample design resulting from the sample redesign of the NHIS New sample design resulting from the sample redesign of the NHIS in 2006 in 2006 – MEPS Panel 12 fielded in January 2007 was the 1st Panel to be MEPS Panel 12 fielded in January 2007 was the 1st Panel to be
selected based on the new NHIS sampleselected based on the new NHIS sample– New NHIS sample design conceptually very similar to the 1995-New NHIS sample design conceptually very similar to the 1995-
2005 design: complex area probability sample 2005 design: complex area probability sample
– Changes in 2006 NHIS design that affected 2007 MEPSChanges in 2006 NHIS design that affected 2007 MEPS Sampling PSUs and SSUs independent of those sampled under the Sampling PSUs and SSUs independent of those sampled under the
previous designprevious design
Previously only HHs with Black and Hispanic persons were Previously only HHs with Black and Hispanic persons were oversampled - oversampled - New New – oversampling of Asian persons – oversampling of Asian persons
Goals of this researchGoals of this research
Evaluate if any significant impact Evaluate if any significant impact on MEPS on MEPS estimates of insurance coverage as a result of estimates of insurance coverage as a result of the dual survey changes implemented in Panel the dual survey changes implemented in Panel 12 which began January 200712 which began January 2007
Evaluate the effectiveness of the MEPS Evaluate the effectiveness of the MEPS nonresponse adjustment strategies in reducing nonresponse adjustment strategies in reducing potential biaspotential bias
Two evaluation approachesTwo evaluation approaches
1. Internal survey comparisons1. Internal survey comparisons– takes advantage of MEPS overlapping panel designtakes advantage of MEPS overlapping panel design
compare the two individual panel specific estimates of health compare the two individual panel specific estimates of health insurance coverage for 2007 and other yearsinsurance coverage for 2007 and other years
– conduct multivariate analysis to determine if panel is a conduct multivariate analysis to determine if panel is a significant predictor of insurance status after controlling significant predictor of insurance status after controlling for other independent variablesfor other independent variables
2. External comparisons2. External comparisons– takes advantage of MEPS linkage to NHIS surveytakes advantage of MEPS linkage to NHIS survey– after MEPS household nonresponse and survey after MEPS household nonresponse and survey
attrition, can insurance estimates for the sampling frame attrition, can insurance estimates for the sampling frame source (NHIS) be replicated using just the MEPS survey source (NHIS) be replicated using just the MEPS survey respondents and the MEPS nonresponse adjusted respondents and the MEPS nonresponse adjusted weights?weights?
Sources of dataSources of data
2007 MEPS compared to 2004-20062007 MEPS compared to 2004-2006– Panel specific estimates within each calendar year Panel specific estimates within each calendar year
(using panel specific weights)(using panel specific weights)
2003-2006 NHIS data2003-2006 NHIS data
Evaluation Approach 1:Evaluation Approach 1:Internal MEPS ComparisonInternal MEPS Comparison
Step 1: Panel specific estimates within yearsStep 1: Panel specific estimates within years
Step 2: Multivariate analysisStep 2: Multivariate analysis
Step 1: Step 1: Evaluation of concordance of Evaluation of concordance of panel specific health insurance coverage panel specific health insurance coverage estimatesestimates
Focus is on 2007 coverage estimatesFocus is on 2007 coverage estimates Panel 12 – dual survey changesPanel 12 – dual survey changes Panel 11 – original sample design and Panel 11 – original sample design and
original DOS-based survey instrumentoriginal DOS-based survey instrument
MEPS annual estimation weightsMEPS annual estimation weights
Each panel is weighted separately and then Each panel is weighted separately and then combinedcombined
Panel specific weight is a function of:Panel specific weight is a function of:– NHIS base weightNHIS base weight
– Poststratification to NHIS full samplePoststratification to NHIS full sample
– Nonresponse adjustment for dwelling unit Nonresponse adjustment for dwelling unit nonresponse and survey attrition within year 1 nonresponse and survey attrition within year 1 and year 2and year 2
– Final raking adjustment to CPS population Final raking adjustment to CPS population control totalscontrol totals
Evaluating the joint effect of the Evaluating the joint effect of the dual survey design changes in 2007dual survey design changes in 2007
Comparison of panel specific national health Comparison of panel specific national health insurance coverage estimates for the insurance coverage estimates for the population under age 65population under age 65
Measures evaluated: Measures evaluated: – 1) some private coverage during the year 1) some private coverage during the year
– 2) public-only coverage during the year 2) public-only coverage during the year
– 3) full year uninsured 3) full year uninsured
Estimates of health insurance coverage for the Estimates of health insurance coverage for the civilian non-institutionalized populationcivilian non-institutionalized population<65 years of age by panel within year<65 years of age by panel within year
CalendarCalendar CoverageCoverage Year 2 in PanelYear 2 in Panel Year 1 in PanelYear 1 in Panel
YearYear MeasureMeasure %% (SE)(SE) %% (SE)(SE)
20072007 Any privateAny private 69.269.2 (0.83)(0.83) 68.768.7 (1.04)(1.04)
Public onlyPublic only 15.815.8 (0.61)(0.61) 15.815.8 (0.73)(0.73)
UninsuredUninsured 15.015.0 (0.53)(0.53) 15.515.5 (0.68)(0.68)
20062006 Any privateAny private 69.969.9 (0.81)(0.81) 70.970.9 (0.78)(0.78)
Public onlyPublic only 15.615.6 (0.56)(0.56) 15.015.0 (0.56)(0.56)
UninsuredUninsured 14.514.5 (0.51)(0.51) 14.114.1 (0.48)(0.48)
20052005 Any privateAny private 70.770.7 (0.83)(0.83) 70.870.8 (0.72)(0.72)
Public onlyPublic only 15.315.3 (0.62)(0.62) 15.115.1 (0.53)(0.53)
UninsuredUninsured 14.014.0 (0.48)(0.48) 14.214.2 (0.47)(0.47)
20042004 Any privateAny private 70.570.5 (0.85)(0.85) 71.771.7 (0.80)(0.80)
Public onlyPublic only 15.215.2 (0.60)(0.60) 14.414.4 (0.57)(0.57)
UninsuredUninsured 14.314.3 (0.49)(0.49) 13.913.9 (0.51)(0.51)
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07
Step 2: Multivariate analysisStep 2: Multivariate analysis
Logistic regression analysis to test for a Logistic regression analysis to test for a panel effect controlling for other panel effect controlling for other explanatory variables related to health explanatory variables related to health insurance coverageinsurance coverage
Separate models by year: 2004-2007Separate models by year: 2004-2007
Covariates included in the modelsCovariates included in the models
Panel classificationPanel classification SexSex Race/ethnicityRace/ethnicity Self-reported health statusSelf-reported health status Marital statusMarital status EducationEducation IncomeIncome MSA, Census regionMSA, Census region Total healthcare expensesTotal healthcare expenses
20072007 20062006 20052005 20042004
Degrees of Degrees of FreedomFreedom
Wald FWald F P-valueP-value Wald FWald F P-valueP-value Wald FWald F P-valueP-value Wald FWald F P-valueP-value
Overall ModelOverall Model 2626 94.8494.84 <0.001<0.001 120.92120.92 <0.001<0.001 101.55101.55 <0.001<0.001 84.2184.21 <0.001<0.001
Model minus interceptModel minus intercept 2525 42.6642.66 <0.001<0.001 66.9266.92 <0.001<0.001 50.9150.91 <0.001<0.001 50.2850.28 <0.001<0.001
Panel classificationPanel classification 11 1.791.79 0.18140.1814 0.060.06 0.81350.8135 0.340.34 0.56270.5627 0.030.03 0.86440.8644
SexSex 11 71.5771.57 <0.001<0.001 66.0966.09 <0.001<0.001 75.2475.24 <0.001<0.001 69.6869.68 <0.001<0.001
Race/ethnicityRace/ethnicity 33 53.8353.83 <0.001<0.001 42.4242.42 <0.001<0.001 56.0356.03 <0.001<0.001 47.5747.57 <0.001<0.001
Health statusHealth status 44 2.152.15 0.07370.0737 2.722.72 0.03020.0302 1.181.18 0.31940.3194 4.904.90 0.00080.0008
Marital statusMarital status 44 14.8414.84 <0.001<0.001 13.0213.02 <0.001<0.001 18.2118.21 <0.001<0.001 26.2626.26 <0.001<0.001
Highest year of educationHighest year of education 44 12.3112.31 <0.001<0.001 32.7132.71 <0.001<0.001 15.3715.37 <0.001<0.001 12.6912.69 <0.001<0.001
Poverty statusPoverty status 44 69.9669.96 <0.001<0.001 103.81103.81 <0.001<0.001 71.1471.14 <0.001<0.001 79.5379.53 <0.001<0.001
RegionRegion 33 21.0021.00 <0.001<0.001 9.019.01 <0.001<0.001 11.9611.96 <0.001<0.001 17.5817.58 <0.001<0.001
Total health care Total health care expendituresexpenditures
11 34.8834.88 <0.001<0.001 68.4468.44 <0.001<0.001 32.4732.47 <0.001<0.001 35.6835.68 <0.001<0.001
-2 x Normalized Log--2 x Normalized Log-Likelihood Full Model:Likelihood Full Model:
13,422.913,422.9 14,319.714,319.7 14,223.514,223.5 14,052.514,052.5
Pseudo Model R-Square:Pseudo Model R-Square: 0.15500.1550 0.15880.1588 0.15160.1516 0.15690.1569
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07
Note: In logistic regression analysis, Y=1 specifies full year uninsured, Y=0 specifies otherNote: In logistic regression analysis, Y=1 specifies full year uninsured, Y=0 specifies other
Final logistic regression model of the Final logistic regression model of the uninsureduninsured, , ages 18-64 years; testing for panel effectages 18-64 years; testing for panel effect
20072007 20062006 20052005 20042004
Degrees of Degrees of FreedomFreedom
Wald FWald F P-valueP-value Wald FWald F P-valueP-value Wald FWald F P-valueP-value Wald FWald F P-valueP-value
Overall ModelOverall Model 2626 105.10105.10 <0.001<0.001 140.17140.17 <0.001<0.001 108.79108.79 <0.001<0.001 100.36100.36 <0.001<0.001
Model minus interceptModel minus intercept 2525 89.8389.83 <0.001<0.001 130.70130.70 <0.001<0.001 105.91105.91 <0.001<0.001 92.4892.48 <0.001<0.001
Panel classificationPanel classification 11 2.642.64 0.10480.1048 2.392.39 0.12340.1234 0.310.31 0.57630.5763 0.510.51 0.47490.4749
SexSex 11 17.3717.37 <0.001<0.001 10.7310.73 0.00120.0012 13.4213.42 0.00030.0003 28.8128.81 <0.001<0.001
Race/ethnicityRace/ethnicity 33 41.3441.34 <0.001<0.001 34.1434.14 <0.001<0.001 49.2249.22 <0.001<0.001 45.7345.73 <0.001<0.001
Health statusHealth status 44 20.3620.36 <0.001<0.001 31.8431.84 <0.001<0.001 32.9632.96 <0.001<0.001 21.9121.91 <0.001<0.001
Marital statusMarital status 44 34.0834.08 <0.001<0.001 28.2628.26 <0.001<0.001 33.9533.95 <0.001<0.001 37.6537.65 <0.001<0.001
Highest year of educationHighest year of education 44 41.5641.56 <0.001<0.001 76.0676.06 <0.001<0.001 64.0764.07 <0.001<0.001 58.3258.32 <0.001<0.001
Poverty statusPoverty status 44 245.13245.13 <0.001<0.001 308.26308.26 <0.001<0.001 261.46261.46 <0.001<0.001 282.70282.70 <0.001<0.001
RegionRegion 33 3.703.70 0.01190.0119 3.993.99 0.00850.0085 0.490.49 0.69060.6906 2.092.09 0.10270.1027
Total health care Total health care expendituresexpenditures
11 19.9419.94 <0.001<0.001 57.9457.94 <0.001<0.001 30.6330.63 <0.001<0.001 34.5134.51 <0.001<0.001
-2 x Normalized Log--2 x Normalized Log-Likelihood Full Model:Likelihood Full Model:
14555.514555.5 15,608.215,608.2 15,268.015,268.0 15,354.915,354.9
Pseudo Model R-Square:Pseudo Model R-Square: 0.28370.2837 0.28540.2854 0.29180.2918 0.28830.2883
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07
Note: In logistic regression analysis, Y=1 specifies some private insurance in year, Y=0 specifies otherNote: In logistic regression analysis, Y=1 specifies some private insurance in year, Y=0 specifies other
Final logistic regression model of the Final logistic regression model of the privatelyprivately insuredinsured, ages 18-64 years; testing for panel effect, ages 18-64 years; testing for panel effect
Evaluation Approach 2: External Evaluation Approach 2: External EvaluationEvaluation
Takes advantage of the linkage of the MEPS to the NHISTakes advantage of the linkage of the MEPS to the NHIS
National estimates of health insurance are derived from National estimates of health insurance are derived from NHIS (NHIS insurance variables) for two analytical samples:NHIS (NHIS insurance variables) for two analytical samples:– Total NHIS sample Total NHIS sample →→ NHIS weights NHIS weights– Reduced NHIS sample obtained from matching with Reduced NHIS sample obtained from matching with
MEPS full year respondents MEPS full year respondents →→ MEPS weights as MEPS weights as adjusted for MEPS nonresponseadjusted for MEPS nonresponse
Since this analysis examines only NHIS insurance data, the Since this analysis examines only NHIS insurance data, the estimates are not impacted by the MEPS CAPI redesignestimates are not impacted by the MEPS CAPI redesign– Thus, assessment of joint impact of the MEPS sample Thus, assessment of joint impact of the MEPS sample
redesign and effectiveness of MEPS nonresponse redesign and effectiveness of MEPS nonresponse adjustments.adjustments.
MEPS-NHIS linked analysisMEPS-NHIS linked analysis
The following 4 NHIS measures of health insurance The following 4 NHIS measures of health insurance coverage were examined:coverage were examined:
any coverageany coverage at the time of the interview (covered; not at the time of the interview (covered; not covered; refused/not ascertained/DK)covered; refused/not ascertained/DK)
private coverageprivate coverage at the time of the interview at the time of the interview (mentioned; not mentioned; refused/not (mentioned; not mentioned; refused/not ascertained/DK)ascertained/DK)
Medicaid coverageMedicaid coverage at the time of the interview at the time of the interview (mentioned; not mentioned; refused/not (mentioned; not mentioned; refused/not ascertained/DK)ascertained/DK)
No health insurance coverageNo health insurance coverage for more than one year for more than one year (yes, no; refused/not ascertained/DK)(yes, no; refused/not ascertained/DK)
NHIS VariableNHIS Variable Using the MEPS Full Year panel Using the MEPS Full Year panel specific weight for calendar year specific weight for calendar year
respondentsrespondents
Using the full sample NHIS and Using the full sample NHIS and NHIS weight (prior year)NHIS weight (prior year)
Calendar YearCalendar Year NOTCOV (Health NOTCOV (Health insurance insurance coverage status)coverage status) %% SESE %% SESE
20072007 Not coveredNot covered 15.315.3** (0.69)(0.69) 16.816.8 (0.29)(0.29)
CoveredCovered 83.783.7** (0.71)(0.71) 82.082.0 (0.29)(0.29)
Refused/NA/DKRefused/NA/DK 1.11.1 (0.18)(0.18) 1.21.2 (0.08)(0.08)
20062006 Not coveredNot covered 15.915.9 (0.53)(0.53) 16.316.3 (0.24)(0.24)
CoveredCovered 83.383.3 (0.56)(0.56) 82.882.8 (0.24)(0.24)
Refused/NA/DKRefused/NA/DK 0.70.7 (0.12)(0.12) 1.01.0 (0.05)(0.05)
20052005 Not coveredNot covered 16.516.5 (0.54)(0.54) 16.216.2 (0.23)(0.23)
CoveredCovered 82.782.7 (0.56)(0.56) 82.882.8 (0.24)(0.24)
Refused/NA/DKRefused/NA/DK 0.80.8 (0.14)(0.14) 1.01.0 (0.06)(0.06)
20042004 Not coveredNot covered 15.715.7 (0.60)(0.60) 16.316.3 (0.26)(0.26)
CoveredCovered 83.483.4 (0.61)(0.61) 82.682.6 (0.27)(0.27)
Refused/NA/DKRefused/NA/DK 0.90.9 (0.14)(0.14) 1.11.1 (0.06)(0.06)
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07
*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level
NHIS NHIS any insurance coverageany insurance coverage at time of interview at time of interview derived from MEPS sample and estimation weights derived from MEPS sample and estimation weights compared to NHIS sample and weights, age <65compared to NHIS sample and weights, age <65
NHIS NHIS private insuranceprivate insurance derived from MEPS derived from MEPS sample and estimation weights compared to sample and estimation weights compared to NHIS sample and weights, age <65NHIS sample and weights, age <65
NHIS VariableNHIS Variable Using the MEPS Full Year panel Using the MEPS Full Year panel specific weight for calendar year specific weight for calendar year
respondentsrespondents
Using the full sample NHIS andUsing the full sample NHIS and
NHIS weight (prior year)NHIS weight (prior year)
Calendar YearCalendar Year HIKINDA (Private HIKINDA (Private insurance)insurance) %% SESE %% SESE
20072007 MentionedMentioned 68.968.9** (1.00)(1.00) 66.366.3 (0.44)(0.44)
Not MentionedNot Mentioned 30.130.1** (1.01)(1.01) 32.632.6 (0.44)(0.44)
Refused/NA/DKRefused/NA/DK 1.11.1 (0.18)(0.18) 1.21.2 (0.08)(0.08)
20062006 MentionedMentioned 69.269.2 (0.81)(0.81) 68.368.3 (0.39)(0.39)
Not MentionedNot Mentioned 30.030.0 (0.80)(0.80) 30.730.7 (0.39)(0.39)
Refused/NA/DKRefused/NA/DK 0.70.7 (0.12)(0.12) 1.01.0 (0.05)(0.05)
20052005 MentionedMentioned 69.469.4 (0.80)(0.80) 68.668.6 (0.39)(0.39)
Not MentionedNot Mentioned 29.829.8 (0.80)(0.80) 30.430.4 (0.39)(0.39)
Refused/NA/DKRefused/NA/DK 0.80.8 (0.14)(0.14) 1.01.0 (0.06)(0.06)
20042004 MentionedMentioned 70.470.4 (0.85)(0.85) 69.269.2 (0.39)(0.39)
Not MentionedNot Mentioned 28.728.7 (0.84)(0.84) 29.729.7 (0.39)(0.39)
Refused/NA/DKRefused/NA/DK 0.90.9 (0.14)(0.14) 1.11.1 (0.06)(0.06)
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07
*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level
NHIS NHIS Medicaid coverageMedicaid coverage derived from MEPS derived from MEPS sample and estimation weights compared to NHIS sample and estimation weights compared to NHIS sample and weights, age <65sample and weights, age <65
NHIS VariableNHIS Variable Using the MEPS Full Year panel Using the MEPS Full Year panel specific weight for calendar year specific weight for calendar year
respondentsrespondents
Using the full sample NHIS and Using the full sample NHIS and
NHIS weight (prior year)NHIS weight (prior year)
Calendar YearCalendar Year HIKINDD HIKINDD (Medicaid)(Medicaid)
%% SESE %% SESE
20072007 MentionedMentioned 8.58.5 (0.66)(0.66) 9.29.2 (0.25)(0.25)
Not MentionedNot Mentioned 90.490.4 (0.66)(0.66) 89.789.7 (0.26)(0.26)
Refused/NA/DKRefused/NA/DK 1.11.1 (0.18)(0.18) 1.21.2 (0.08)(0.08)
20062006 MentionedMentioned 8.28.2 (0.41)(0.41) 8.88.8 (0.21)(0.21)
Not MentionedNot Mentioned 91.091.0 (0.42)(0.42) 90.390.3 (0.21)(0.21)
Refused/NA/DKRefused/NA/DK 0.70.7 (0.12)(0.12) 1.01.0 (0.05)(0.05)
20052005 MentionedMentioned 7.97.9 (0.46)(0.46) 8.58.5 (0.20)(0.20)
Not MentionedNot Mentioned 91.391.3 (0.47)(0.47) 90.590.5 (0.20)(0.20)
Refused/NA/DKRefused/NA/DK 0.80.8 (0.14)(0.14) 1.01.0 (0.06)(0.06)
20042004 MentionedMentioned 7.87.8 (0.38)(0.38) 8.68.6 (0.21)(0.21)
Not MentionedNot Mentioned 91.491.4 (0.40)(0.40) 90.390.3 (0.22)(0.22)
Refused/NA/DKRefused/NA/DK 0.90.9 (0.14)(0.14) 1.11.1 (0.06)(0.06)
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07
*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level
NHIS NHIS insurance coverage for >1 yearinsurance coverage for >1 year derived from derived from MEPS sample and estimation weights compared MEPS sample and estimation weights compared to NHIS sample and weights, age <65to NHIS sample and weights, age <65
NHIS VariableNHIS Variable Using the MEPS Full Year panel Using the MEPS Full Year panel specific weight for calendar year specific weight for calendar year
respondentsrespondents
Using the full sample NHIS and Using the full sample NHIS and NHIS weight (prior year)NHIS weight (prior year)
Calendar YearCalendar Year HILAST (No HILAST (No health insurance health insurance for more than for more than one year)one year) %% SESE %% SESE
20072007 MentionedMentioned 10.710.7 (0.61)(0.61) 11.511.5 (0.24)(0.24)
Not MentionedNot Mentioned 89.089.0 (0.61)(0.61) 87.987.9 (0.25)(0.25)
Refused/NA/DKRefused/NA/DK 0.20.2 (0.05)(0.05) 0.50.5 (0.04)(0.04)
20062006 MentionedMentioned 10.710.7 (0.43)(0.43) 11.111.1 (0.20)(0.20)
Not MentionedNot Mentioned 88.888.8 (0.45)(0.45) 88.388.3 (0.21)(0.21)
Refused/NA/DKRefused/NA/DK 0.50.5 (0.12)(0.12) 0.60.6 (0.04)(0.04)
20052005 MentionedMentioned 11.211.2 (0.43)(0.43) 11.111.1 (0.18)(0.18)
Not MentionedNot Mentioned 88.588.5 (0.44)(0.44) 88.488.4 (0.19)(0.19)
Refused/NA/DKRefused/NA/DK 0.30.3 (0.06)(0.06) 0.50.5 (0.04)(0.04)
20042004 MentionedMentioned 10.310.3 (0.45)(0.45) 10.810.8 (0.21)(0.21)
Not MentionedNot Mentioned 89.389.3 (0.46)(0.46) 88.688.6 (0.21)(0.21)
Refused/NA/DKRefused/NA/DK 0.40.4 (0.09)(0.09) 0.60.6 (0.05)(0.05)
Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07Source: Medical Expenditure Panel Survey Household Component (MEPS-HC), 2004-07; National Health Interview Survey (NHIS), 2004-07
*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level*significant difference in MEPS derived NHIS estimate relative to the NHIS derived estimate at the .05 level
What did we learn?What did we learn?
MEPS overlapping panel design facilitates the MEPS overlapping panel design facilitates the evaluation of periodic survey design changes. evaluation of periodic survey design changes.
The linkage of MEPS and NHIS also facilitates data The linkage of MEPS and NHIS also facilitates data quality assessments.quality assessments.
The internal comparison of MEPS insurance The internal comparison of MEPS insurance estimates (<65) by panel for 2007 and for years estimates (<65) by panel for 2007 and for years prior to the redesign did not show any significant prior to the redesign did not show any significant differences.differences.
The logistic regression analysis likewise did not The logistic regression analysis likewise did not reveal any significant effect for Panel.reveal any significant effect for Panel.
What did we learn (cont.)?What did we learn (cont.)?
Phase 2 of this study evaluated the joint impact of the new Phase 2 of this study evaluated the joint impact of the new 2007 sample design and the survey’s post-survey 2007 sample design and the survey’s post-survey adjustments.adjustments.
The MEPS-NHIS linked analysis only showed modest The MEPS-NHIS linked analysis only showed modest differences in 2 of 4 NHIS insurance variables examined.differences in 2 of 4 NHIS insurance variables examined.
No evidence of nonresponse bias attributable to year 1 No evidence of nonresponse bias attributable to year 1 survey attrition.survey attrition.
Limitation of this study: We could not tease out the Limitation of this study: We could not tease out the independent effects of the two survey modifications.independent effects of the two survey modifications.
Nevertheless, the dual survey changes in 2007 did not Nevertheless, the dual survey changes in 2007 did not appear to have a major impact on insurance coverage appear to have a major impact on insurance coverage estimates.estimates.
Future researchFuture research
Assess the 2008 MEPS panel specific estimates Assess the 2008 MEPS panel specific estimates CAPI platform and sample design will be the same in CAPI platform and sample design will be the same in
both individual overlapping panels. both individual overlapping panels.
Continued evaluations of variables used to adjust for Continued evaluations of variables used to adjust for nonresponse.nonresponse.
Evaluate transitions in insurance coverage from NHIS to Evaluate transitions in insurance coverage from NHIS to MEPS, pre-and post-new design.MEPS, pre-and post-new design.
Evaluate trends in transitions in coverage in MEPS from Evaluate trends in transitions in coverage in MEPS from year 1 to year 2, pre- and post new design.year 1 to year 2, pre- and post new design.
Review CAPI programming and edit specifications. Review CAPI programming and edit specifications.
DiscussionDiscussion
Comments on the internal and external Comments on the internal and external evaluation strategies used in this study? evaluation strategies used in this study?
What other strategies have been (or can be) What other strategies have been (or can be) used to evaluate survey data quality used to evaluate survey data quality following implementation of design or following implementation of design or methodological modifications?methodological modifications?