View
226
Download
0
Category
Preview:
Citation preview
7/24/2019 Our Survey Methodology in Detail.docx
1/21
Our Survey Methodology in Detail
Sampling
The typical Pew Research Center for the People & the Press national survey selects a random
digit sample of both landline and cell phone numbers in all 50 U.. states and the !istrict of
Columbia. "s the proportion of "mericans who rely solely or mostly on cell phones for theirtelephone service continues to grow# sampling both landline and cell phone numbers helps to
ensure that our surveys represent all adults who have access to either $only about % of
households in the U.. do not have access to any phone'. (e sample landline and cell phonenumbers to yield a combined sample with appro)imately *5 of the interviews conducted by
landline and +5 by cell phone. This ratio is based on an analysis that attempts to balance cost
and fieldwor, considerations as well as to improve the overall demographic composition of the
sample $in terms of age# race-ethnicity and education'. This ratio also ensures an
adeuate number of cell only respondents in each survey.
The design of the landline sample ensures representation of both listed and unlisted numbers
$including those not yet listed' by using random digit dialing. This method uses random
generation of the last two digits of telephone numbers selected on the basis of the area code#telephone e)change# and ban, number. " ban, is defined as /00 contiguous telephone numbers#
for e)ample 0015551/%00 to 0015551/%22. The telephone e)changes are selected to be
proportionally stratified by county and by telephone e)change within the county. That is# thenumber of telephone numbers randomly sampled from within a given county is proportional to
that county3s share of telephone numbers in the U.. 4nly ban,s of telephone numbers
containing one or more listed residential numbers are selected.
The cell phone sample is drawn through systematic sampling from dedicated wireless ban,s of/00 contiguous numbers and shared service ban,s with no directory1listed landline numbers $to
ensure that the cell phone sample does not include ban,s that are also included in the landline
sample'. The sample is designed to be representative both geographically and by large and smallwireless carriers $also see cell phonesfor more information'.
oth the landline and cell samples are released for interviewing in replicates# which are small
random samples of each larger sample. Using replicates to control the release of telephone
numbers ensures that the complete call procedures are followed for all numbers dialed. The useof replicates also improves the overall representativeness of the survey by helping to ensure that
the regional distribution of numbers called is appropriate.
(hen interviewers reach someone on a landline phone# they randomly as, half the sample if they
could spea, with 6the youngest male# / years of age or older# who is now at home7 and theother half of the sample to spea, with the youngest female# / years of age or older# who is now
at home.7 8f there is no eligible person of the reuested gender currently at home# interviewers
as, to spea, with the youngest adult of the opposite gender# who is now at home. This method of
http://www.people-press.org/methodology/sampling/cell-phones/http://www.people-press.org/methodology/sampling/cell-phones/http://www.people-press.org/methodology/sampling/cell-phones/7/24/2019 Our Survey Methodology in Detail.docx
2/21
selecting respondents within each household improves participation among young people who
are often more difficult to interview than older people because of their lifestyles.
Unli,e a landline phone# a cell phone is assumed in Pew Research polls to be a personal device.8nterviewers as, if the person who answers the cell phone is / years of age or older to
determine if the person is eligible to complete the survey $also seecell phone surveysfor moreinformation'. This means that# for those in the cell sample# no effort is made to give other
household members a chance to be interviewed. "lthough some people share cell phones# it isstill uncertain whether the benefits of sampling among the users of a shared cell phone outweigh
the disadvantages.
ampling error results from collecting data from some# rather than all# members of thepopulation. 9or each of our surveys# we report a margin of sampling error for the total sample
and usually for ,ey subgroups analy:ed in the report $e.g.# registered voters# !emocrats#
Republicans# etc.'. 9or e)ample# the sampling error for a typical Pew Research Center for the
People & the Press national survey of /#500 completed interviews is plus or minus %.2
percentage points with a 25 confidence interval. This means that in 25 out of every /00samples of the same si:e and type# the results we obtain would vary by no more than plus or
minus %.2 percentage points from the result we would get if we could interview every member ofthe population. Thus# the chances are very high $25 out of /00' that any sample we draw will be
within * points of the true population value. The sampling errors we report also ta,e into account
the effect of weighting. $"lso see (hy probability samplingfor more information'
Nonresponse
"t least ; attempts are made to complete an interview at every sampled telephone number. The
calls are staggered over times of day and days of the wee, $including at least one daytime call' to
ma)imi:e the chances of ma,ing contact with a potential respondent. 8nterviewing is also spreadas evenly as possible across the field period. "n effort is made to recontact most interview
brea,offs and refusals to attempt to convert them to completed interviews.
Response rates for Pew Research polls typically range from 5 to /5< these response rates arecomparable to those for other ma=or opinion polls. The response rate is the percentage of ,nown
or assumed residential households for which a completed interview was obtained. The response
rate we report is computed using the "merican "ssociation for Public 4pinion Research3s$""P4R' Response Rate * $RR*' method $9or a full discussion of response rates see ""P4R3s
tandard !efinitions'. 9ortunately# low response rates are not necessarily an indication of
nonresponse bias as we discuss in the problem of declining response rates.
8n addition to the response rate# we sometimes report the contact rate# cooperation rate# or thecompletion rate for a survey. The contact rate is the proportion of wor,ing numbers where a
reuest for an interview was made. The cooperation rate is the proportion of contacted numbers
where someone gave initial consent to be interviewed. The completion rate is the proportion ofinitially cooperating and eligible households where someone completed the interview.
Data weighting
http://www.people-press.org/methodology/methodology/collecting-survey-data/cell-phone-surveys/http://www.people-press.org/methodology/methodology/collecting-survey-data/cell-phone-surveys/http://www.people-press.org/methodology/sampling/why-probability-sampling/http://www.aapor.org/Standard_Definitions1.htmhttp://www.aapor.org/Standard_Definitions1.htmhttp://www.people-press.org/methodology/collecting-survey-data/the-problem-of-declining-response-rates/http://www.people-press.org/methodology/collecting-survey-data/the-problem-of-declining-response-rates/http://www.people-press.org/methodology/methodology/collecting-survey-data/cell-phone-surveys/http://www.people-press.org/methodology/sampling/why-probability-sampling/http://www.aapor.org/Standard_Definitions1.htmhttp://www.aapor.org/Standard_Definitions1.htmhttp://www.people-press.org/methodology/collecting-survey-data/the-problem-of-declining-response-rates/7/24/2019 Our Survey Methodology in Detail.docx
3/21
>onresponse in telephone interview surveys can produce biases in survey1derived estimates.
urvey participation tends to vary for different subgroups of the population# and these subgroups
are li,ely to also vary on uestions of substantive interest. To compensate for these ,nownbiases# the sample data are weighted for analysis.
The landline sample is first weighted by household si:e to account for the fact that people inlarger households have a lower probability of being selected. 8n addition# the combined landline
and cell phone sample is weighted to account for the fact that respondents with both a landlineand cell phone have a greater probability of being included in the sample.
The sample is then weighted using population parameters from the U.. Census ureau for
adults / years of age or older. The population parameters used for weighting are? gender by age#gender by education# age by education# region# race and @ispanic origin that includes a brea, for
@ispanics based on whether they were born in the U.. or not# population density and among
non1@ispanic whites A age# education and region. The parameters for these variables are from the
Census ureau3s %0/% "merican Community uvey $e)cluding those in institutionali:ed group
uarters'# e)cept for the parameter for population density which is from the %0/0 Census. Thesepopulation parameters are compared with the sample characteristics to construct the weights. 8n
addition to the demographic parameters# the sample is also weighted to match current patterns oftelephone status $landline only# cell phone only or both landline and cell phone'# based on
e)trapolations from the %0/B >ational @ealth 8nterview urvey. The final weights are derived
using an iterative techniue that simultaneously balances the distributions of all weightingparameters. ou can view the demographic and phone usage uestions we use to compare the
sample characteristics to our weighting parametershere.
(eighting cannot eliminate every source of nonresponse bias. >onetheless# properly1conducted
public opinion polls have a good record in achieving unbiased samples. 8n particular# election
polling A where a comparison of the polls with the actual election results provides an opportunityto validate the survey results A has been very accurate over the years $see the >ational Council
on Public Polls Dvaluations of the %0/% and %0/0Dlections'.
Data analysis
Dach Pew Research survey report includes a 6topline uestionnaire7 with all of the uestions
from that survey with the e)act uestion wording and response options as they were read torespondents. This topline provides the results from the current survey for each uestion# as well
as results from previous surveys in which the same or similar uestions were as,ed.
9or discussion of the results in reports and commentaries# differences among groups are reportedwhen we have determined that the relationship is statistically significant and therefore is unli,elyto occur by chance. tatistical tests of significance ta,e into account the effect of weighting. 8n
addition# to support any causal relationships discussed# more advanced multivariate statistical
modeling techniues are often employed to test whether these connections e)ist# although theresults of these models may or may not be shown in the actual report.
http://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.ncpp.org/?q=node/137http://www.ncpp.org/?q=node/134http://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.ncpp.org/?q=node/137http://www.ncpp.org/?q=node/1347/24/2019 Our Survey Methodology in Detail.docx
4/21
9or most studies# it is our policy to release datasets from Pew Research surveys five months after
the data was collected and archive them on our website as uic,ly as possible. Please visit our
datasetspage for further information.
http://www.people-press.org/category/datasets/http://www.people-press.org/category/datasets/7/24/2019 Our Survey Methodology in Detail.docx
5/21
7/24/2019 Our Survey Methodology in Detail.docx
6/21
7/24/2019 Our Survey Methodology in Detail.docx
7/21
7/24/2019 Our Survey Methodology in Detail.docx
8/21
7/24/2019 Our Survey Methodology in Detail.docx
9/21
7/24/2019 Our Survey Methodology in Detail.docx
10/21
7/24/2019 Our Survey Methodology in Detail.docx
11/21
7/24/2019 Our Survey Methodology in Detail.docx
12/21
7/24/2019 Our Survey Methodology in Detail.docx
13/21
7/24/2019 Our Survey Methodology in Detail.docx
14/21
7/24/2019 Our Survey Methodology in Detail.docx
15/21
7/24/2019 Our Survey Methodology in Detail.docx
16/21
7/24/2019 Our Survey Methodology in Detail.docx
17/21
7/24/2019 Our Survey Methodology in Detail.docx
18/21
7/24/2019 Our Survey Methodology in Detail.docx
19/21
7/24/2019 Our Survey Methodology in Detail.docx
20/21
7/24/2019 Our Survey Methodology in Detail.docx
21/21
Recommended