View
216
Download
0
Category
Tags:
Preview:
Citation preview
Comparing ACS with Census Sample and Current Survey Data
Partnership and Data Services Training Susan Love
April 12, 2005
Some Sources for Comparison
Census long form samples in general and Census 2000 in particular
Current Population Survey and Annual Social and Economic Supplement
American Housing Survey
What should we compare?
Sampling frames and sample designData collection time frames and modesInterviewer requirementsRespondent requirementsData completenessWeighting and estimation methodsFinal results at comparable geographies
ACS and the Decennial Census
How do the samples compare?
Similarities
Systematic samples of MAF addresses
Differential sampling rates for areas based on population size
Sampling Rates
Governmental Unit Size: Number of Occupied Housing Units
Census 2000 Sampling
Rates
ACS 5-year Sampling
Rates
0 - 200 50.0% 50.0%
201 - 800 50.0% ~35.0%
801 - 1200 25.0% ~17.5%
1201 - 2000 16.7% ~12.0%
Over 2000 12.5% ~8.5%
How do the samples compare?
Differences
ACS samples every year and spreads sample over 12 months; census samples once a decade and uses the entire sample at the same time
ACS subsamples for personal visit followup; court ruled against census subsampling
Census sample estimates based on about 18 million housing units; ACS 5 year estimates based on about 11 million housing units
How does data collection compare?
Similarities
Primary method is self-response to a paper questionnaire
Content is the same
How does data collection compare?
Differences
ACS nonresponse FU uses CATI and CAPI instruments; past censuses have used only paper questionnaires
ACS data collected only from household members; census data often collected from neighbors
ACS interviews conducted by experienced and well-supervised staff; census enumerations conducted by inexperienced temporary workforce
What is the impact on the data?
ACS estimates have higher sampling error – measures are released for each estimate and
shown as 90% confidence limits or margins of error in every table
Similar sampling error measures for census long form sample estimates have not been provided
What is the impact on the data?
ACS estimates have lower potential nonsampling error – Unit nonresponse: no data for a case– Item nonresponse: data missing for an item
These and other measures are released on ACS’s Quality Measures website
www.census.gov/acs/www/UseData/sse/index.htm
11
Unit Nonresponse
97.7--2002
96.7--2003
96.7--2001
95.191.22000
ACSCensus 2000 Sample
Response Rates (100-unit nr rate)
12
Item Nonresponse
7.114.9Housing items - Followup
5.911.5Housing items - Mail
4.512.9Population items - Followup
8.18.7Population items - Mail
6.610.4All modes/ items
2000
ACS
Census 2000
Sample
Summary Allocation Rates(amount of data missing from
interviews)
What do the estimates represent?
Most ACS estimates are said to be the average characteristics of an area over a calendar year
Census sample estimates are often said to be characteristics of an area as of Census Day
Is this so?
What do the estimates represent?
Both the ACS interview date and the census enumeration date play important roles
ACS uses the interview date as the single reference point, or as the end of a reference period, for all data collection
The Census 2000 sample did too, except for income, migration, and agricultural sales
Weighted Census 2000 Sample and 2000 ACS Housing Units by Response Week
0
5,000,000
10,000,000
15,000,000
20,000,000
25,000,000
30,000,000
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52
Interview Week in Calendar Year 2000
Wei
gh
ted
Ho
usi
ng
Un
its
Census 2000 Sample 2000 ACS
What do the Census 2000 sample estimates represent?
Census 2000 data collection was a sequential process of 3 operations over a 6 month period
Data from these operations were processed together, weighted and tabulated
The characteristic estimates are the sum of the peaks and valleys of the enumerations, not Census Day
Census 2000 Data Collection Design
M o n th o f C e n s u s 2 0 0 0 E n u m e ra tio n
M a rc h A p r il M a y J u n e J u ly A u g u s t
M a il
N R F U C IF U
What do the ACS estimates represent?
ACS data collection is a sequential process of 3 operations on each of 12 monthly samples over 3 month periods– All 3 operations take place every month on different
sample panels, a continuous series of interviews
Data collected in a calendar year are grouped by interview month; a weighting adjustment is applied to smooth overall monthly differences
Summed estimates are considered the average characteristics of all areas for the calendar year
ACS Data Collection Design
1 9 9 9 2 0 0 0 M o n th o f In te rv ie w
A C S S a m p le P a n e l
N o v e m b e r D e c e m b e r J a n u a ry F e b ru a ry M a rc h A p ril M a y J u n e
N o ve m b e r M a il C A T I C A P I
D e c e m b e r
M a il C A T I C A P I
J a n u a ry
M a il C A T I C A P I
F e b ru a ry
M a il C A T I C A P I
M a rc h
M a il C A T I C A P I
A p ril
M a il C A T I C A P I M a y
M a il C A T I
J u n e
M a il
How did the 2000 ACS and Census 2000 results compare?
Comparisons with Census 2000
Four reports by Bureau staff compared the 2000 ACS data with Census 2000 data at the national level
Two staff reports and four reports by outside researchers compared three-year averages from the 1999-2001 ACS test sites with Census 2000 at county and tract levels
www.census.gov/acs/www/advMeth/Reports
Comparisons with Census 2000
ACS profile distributions were compared to Census 2000 profile distributions without group quarters pop
ACS distributions were surprisingly similar to Census 2000– Only about 9% of the profile table estimates
differed by more than 1 percentage point
Self-response data from mail returns were responsible for the consistent results
Comparisons with Census 2000
Characteristics differing the most – Race and relationship (Table 1)– Ancestry, disability, high school graduates
(Table 2)– Labor force participation, median household and
family income, and poverty rates (Table 3)– Year built, rooms, and house heating fuel
(Table 4)
Summary of National-Level Comparisons of General Demographic and Housing Characteristics (Table 1)
0 5 10 15 20 25 30 35
-1.0% or greater
-0.5% to -0.9%
-0.1% to -0.4%
0.0%
0.1% to 0.4%
0.5% to 0.9%
1.0% or greater
Perc
enta
ge P
oint
Diff
eren
ce(C
2SS
- Cen
sus
2000
)
Percent of National-Level Comparisons (n=94)
General Demographic Characteristics
ACS estimated more White and fewer Some Other Race – experience and training of FRs
ACS estimated fewer married couple families – ACS weighting, edit difference (marital status no longer 100% census item)
Item nonresponse – Race: Census 3.9%, ACS 2.4%. Relationship: Census 2.2%, ACS relationship 1.6%
Summary of National-Level Comparisons of Social Characteristics (Table 2)
0 5 10 15 20 25 30 35 40
-1.0% or greater
-0.5% to -0.9%
-0.1% to -0.4%
0.0%
0.1% to 0.4%
0.5% to 0.9%
1.0% or greater
Perc
enta
ge P
oint
Diff
eren
ce(C
2SS
- Ce
nsus
200
0)
Percent of National-Level Comparisons (n=76)
Social Characteristics
ACS estimated more English, German, and Irish ancestry -- supplementary sample design
ACS estimated more people with high school diploma and no college – time frame, data capture
ACS estimated fewer people with disabilities – poor question design, answered well only in CATI and CAPI
Item nonresponse – Ancestry: Census 19%, ACS 11.7%. Ed. attain.: Census 7.2%, ACS 4.8%. Disability: Census 8.5%, ACS 2.1%
Summary of National-Level Comparisons of Economic Characteristics (Table 3)
0 5 10 15 20 25 30 35 40
-1.0% or greater
-0.5% to -0.9%
-0.1% to -0.4%
0.0%
0.1% to 0.4%
0.5% to 0.9%
1.0% or greater
Perc
enta
ge P
oint
Diff
eren
ce
(C2S
S -
Cens
us 2
000)
Percent of National-Level Comparisons (n=75)
Economic Characteristics
ACS estimated higher labor force participation – slight question and edit difference
ACS estimated more families with young children and all parents in the labor force – weighting and edit differences
ACS estimates a lower median household and family income and higher poverty – data capture
Item nonresponse-- Labor force status: Census 11.1%, ACS 6.0%. Household income: Census 30%, ACS 24%.
Summary of National-Level Comparisons of Housing Characteristics (Table 4)
0 5 10 15 20 25 30 35 40
-1.0% or greater
-0.5% to -0.9%
-0.1% to -0.4%
0.0%
0.1% to 0.4%
0.5% to 0.9%
1.0% or greater
Perc
enta
ge P
oint
Diff
eren
ce
(C2S
S -
Cens
us 2
000)
Percent of National-Level Comparisons (n=81)
Physical Housing Characteristics
ACS estimated more units built in 1939 or before – troublesome question (multi-unit rentals)
ACS estimated more 4 and 5 room units – troublesome question; old edit assumptions
ACS estimated fewer units heated with gas – troublesome question (multi-unit rentals)
Item nonresponse -- Year built: Census 12.7%, ACS 14.9%. Rooms: Census 7.8%, ACS 4.2%. Heating fuel: Census 7.4%, ACS 2.1%
Financial Housing Characteristics
ACS estimated fewer owner-occupied units with a mortgage – edit difference (more info in ACS)
ACS estimated fewer households at the low end and more households at the high end of gross rent as a % of income – higher income in Census 2000 than in ACS (and ASEC)
Item nonresponse– First mortgage: Census 6.0%, ACS 2.0%. Gross rent: Census 37.7%, ACS 20.3%
Summary of Reasons for Differences
Timing and reference periods
Quality of nonresponse followup– Training and staff experience– Treatment of proxy information– Data completeness
Troublesome questions
Weighting
Conclusions
Most estimates are very comparable, and the differences are not unexpected or unexplainable
ACS has higher level of overall response and individual item response, so less chance of nonresponse bias
ACS household information comes only from a member so probably more accurate
ACS is a better way to collect this wide-ranging information than was the decennial census because the distribution of the data over the collection time frame is more meaningful
What about current surveys?
Comparisons with CPS & ASEC
Coming this year --
Industry, Occupation, and Class of Worker
Income and Poverty
Race and Ethnicity
Households and Families
Age and Sex
Comparisons with CPS and ASEC
73,000 addresses in CPS sample and 250,000 address in ACS sample each month; 100,000 address in ASEC each year
CPS interviews about 55,000 households and ACS interviews about 170,000 households each month
CPS units interviewed 8 times in 2 years; ACS and ASEC units interviewed only once
ASEC interviews nearly 78,000 households once a year; ACS interviews about 2 million households over the course of a year
Comparisons with CPS and ASEC
Monthly CPS collects detailed labor force participation and releases the official estimates every month
ASEC covers several topics, releases national and state data yearly, and produces the official national income and poverty estimates
ACS produces data on many of the same topics and releases data for the nation, states, and lower-level geographies with populations of at least 65,000 every year
Comparisons with CPS and ASEC
CPS and ASEC focus on fewer topics and go into them in greater depth than ACS
They produce more complete measurements of concepts; ACS only touches on many topics, as did the census long form samples
ACS produces information for much lower levels of geography because of its larger sample
Comparisons with American Housing Survey (AHS)
Coming next year --
Measuring the financial characteristics of housing– Homeownership – Property value– Selected monthly owner costs– Contract and gross rents
Comparisons with the American Housing Survey
About 64,000 addresses in AHS national sample, with about 55,000 interviews conducted every 2 years; ACS samples nearly 3 million addresses every year, resulting in over 2 million interviews
AHS has visited many of the same sample addresses every 2 years since 1985; ACS visits a different sample every month
Comparisons with the American Housing Survey
AHS collects very detailed information on physical and economic characteristics and conditions of housing and neighborhoods; releases data at national and regional level, and for 6 large metropolitan areas
ACS produces data on some of the same topics but not in detail; releases information at much lower levels of geography
General Guidelines for Single-year Annual Estimates
Use ASEC for national estimates of income and poverty, and ACS for lower levels of geography
Use CPS for national and state estimates of basic labor force participation, and ACS for estimates by socioeconomic characteristics at lower levels of geography
Use AHS for detailed characteristics of housing at the national level and 6 large metro areas, and ACS for lower levels of geography.
ACS Data Products
In 2005 for 2004 ACS
Redesigned Base (detail) table package – over 900
New Subject tables – about 50
New Selected Population Profiles – well over half of the SF-4 groups
Redesigned and expanded tabular profiles
Expanded narrative profiles
In 2006 for 2005 ACS
Further narrative profile expansion
New thematic maps
Illustration of an ACS Release schedule
Type of Data
Population
Size of Area
Data For The Previous Year Released In The Summer Of:
2006 2007 2008 2009 2010 2011 2012
Annual Estimates
65,000+
3-year Estimates
20,000+
5-year Estimates
Down to Census Tract and Block Group
Recommended