Survey Research Slides Prepared by Alison L. OMalley Passer
Chapter 7
Slide 2
Surveys: An Introduction Surveys rely on interviews and
questionnaires to gather info about people What is the last survey
you took? Take Our Survey!
Slide 3
Populations and Samples Operationally defined by sampling frame
Population Sampl e Whats the difference between representative and
nonrepresentative samples? Representative or biased?
Slide 4
Surveys Can we trust the data? Participants responses may be
distorted by social desirability bias, or perhaps theyre
misremembering Response Truth ? Fret not. High quality survey
research is doable!
Slide 5
Selecting the Sample Probability sampling Every member of the
population has chance of being sampled Probability of selection can
be specified Nonprobability sampling Probability sampling
conditions do not apply
Slide 6
Probability Sampling Simple random sampling Build a sampling
frame containing all population members Stratified random sampling
Sampling frame divided into groups (based on demographic
characteristics) Random sampling applied to each group
Slide 7
Probability Sampling Cluster sampling Units (e.g., schools)
containing population members are identified Essentially, this step
creates the sampling frame These clusters are then randomly sampled
May not represent the entire population What if theres no sampling
frame?
Slide 8
Nonprobability Sampling Convenience sampling Grab whomever you
can Likely to generate a nonrepresentative sample Quota sampling
Sample designed to mirror population characteristics (e.g., % of
females) Uses convenience sampling to create sample within each
quota group (e.g., males and females)
Slide 9
Nonprobability Sampling Self-selected samples Participants
elect to participate (as opposed to being sought out by researcher)
A form of convenience sampling Likely to generate a large sample
size, but keep in mind that representativeness matters more than
sample size!
Slide 10
Nonprobability Sampling Purposive sampling Sample created in
line with study goals (e.g., focus only on students in Top 10
graduate programs in research on the work habits of successful
graduate students) Two common strategies Expert sampling Snowball
sampling participants recruit others to participate
Slide 11
Sampling: Check Your Understanding What is the difference
between quota sampling and stratified random sampling?
Slide 12
Margin of Sampling Error Sample results are estimates of the
true population value Sampling variability captures how sample
characteristics fluctuate If you roll two dice 100 times and
encounter this 5 and 2 pattern 13 times, its extremely unlikely
youll see this pattern 13 more times in the next 100 rolls
Slide 13
Margin of Sampling Error Thus, sampling error acknowledges that
our population estimates vary depending on the sample Survey data
are then accompanied by a margin of sampling error, a range of
values within which the true population value falls Keeping in mind
that we can never be 100% certain in our results, we also report
confidence levels (typically 95%)
Slide 14
Sample Reporting For results based on the total sample of
national adults, one can say with 95% confidence that the margin of
sampling error is 2 percentage points. Gallup Economy Poll - May
28, 2013
http://www.gallup.com/poll/162797/economic-confidence-holds-steady-high-
level.aspx
Slide 15
Taking the following data into account, why dont researchers
aim for more precise estimates?
Slide 16
Constructing Questionnaires Write a survey item asking
participants about their exercise habits Now, ask a classmate for
feedback on your item Developing a questionnaire takes a
significant amount of time and effort, and typically several
versions are piloted before the final version is rolled out.
Slide 17
Types of Questions Closed-ended questions provide specific
response options, whereas open-ended questions do not, allowing
participants to answer in whatever form they choose Was your item
about exercise behavior open- or closed-ended? Open-ended questions
generally are more difficult to work with. Why? Open-Ended and
Closed-Ended
Forced choice Rating scales Closed-Ended Question Types
Describe the signature features of Likert response formats.
Slide 20
Question Wording Leading questions Loaded questions
Double-barreled questions Double negatives Common Pitfalls to Avoid
Are you or are you not in favor of terrorists in our country
corrupting our young people and threatening our core values? Help!
Identify whats wrong with this question and rewrite it.
Slide 21
Putting the Survey Together Group related questions together
Place open-ended questions before closed-ended questions Move from
more general to more specific questions Place personally sensitive
questions at or near the end General Rules of Thumb
Slide 22
Administering Surveys Face-to-face (in-person) interviews
Achieve higher response rates Facilitate establishment of rapport
Enable standardized approach Interviewer can clarify any
participant confusion But, theyre pricey! Supplements to ongoing
behavioral observations
Slide 23
Administering Surveys What can interviewers do to aid each of
the following? 1.Limitations in participant memory 2.Response
distortion due to interviewer bias or other interviewer
effects
Slide 24
Other Ways to Administer Surveys Telephone Mail Online Discuss
the strengths and limitations of each mode of data collection.
Slide 25
Additional Survey Considerations Nonresponse bias occurs when
participants who declined to participate would have responded
differently than participants did Introduces more error into
population estimates Although lower response rates do not appear to
drive nonresponse bias, declining participation rates are of
concern Many researchers offer incentives to encourage
participation
Slide 26
What survey design features enable you to have greater
confidence in the results? Be on the lookout for bogus surveys! Be
a Smart Survey Consumer Think Critically