Surveys

The most widely used data-gathering device in sociology

Surveys

Survey research rests on sampling. A sample is simply the collection of

people from whom you are gathering data.

There are tons of ways to sample, but we want to be sure that we sample with generalizabilty in mind.

Surveys What type of sampling does the General

Social Survey use???

In 2004, it was a multistage cluster sample. It is a full probability sample of US households, meaning that each house had an equal chance of being selected.

Persons in households were selected systematically to make demographic characteristics of the sample equivalent to the characteristics of the U.S. population.

Surveys What are the implications of the General Social

Survey’s sampling???

The GSS is an adults-only survey of persons in households. Therefore, it underrepresents: 18 – 24 year-olds (many not living in households—

military, college, roaming) 65 and over (many not living in households—

vacations, RVs, assisted living) Persons who live in large households (only one

person per household is interviewed) Homeless and some poor (not in official households,

in shelters, on streets)

Surveys Sampling is a technique for selecting subjects. So what is a

survey?

A process of presenting a standard series of questions to a sample of persons.

The purpose is to accurately reflect the beliefs, attitudes, and behaviors of the sample, so that one can generalize accurate information to some larger population.

The survey is the most widely used technique in sociology because it is best suited for looking at the complex social world. To capture that world accurately, we have to measure it “in situ.” That means we must sample from it, and measure many different phenomena, where they are found naturally.

Surveys Types of Surveys: The survey can be administered as a(n):

Written document filled out by the sample’s respondents alone in groups or individually in group settings (group

administered) Face-to-face interview, with questions read by an

interviewer Telephone interview Electronic Questionnaire

Surveys Survey Delivery: Written Questionnaires:

Mail Group Administered Questionnaires Drop-off Surveys

Electronic Questionnaires: Distributed as E-mail Web pages—sampling can be done via normal

methods, but often aren’t Public computers in high-traffic areas In the future:

Files sent via mail, replacing written surveys Mixed formats offered to subjects (choose: pen-and-paper, web

page, file on disk)

Surveys How is the General Social Survey Delivered

and Administered???

The GSS is a one-on-one interview. An interviewer asks subjects questions.

In 2000, the interviewer used paper and pencil.

In 2002 they switched to computer assisted personal interviewing (CAPI). A computer program provided questions and interviewers punched in answers, much like in telephone polling with CATI.

Surveys Before Choosing a type of survey, you

should consider four (4) key issues:

1. Respondent Attitude: Think like a subject. How will they react to the way they are approached?

2. Nature of Questions: Open-ended and scale questions would be difficult over the phone. You have to consider whether respondents can adequately answer your questions via your medium.

Surveys Before Choosing a type of survey, you

should consider four (4) key issues:

3. Cost: What can you afford? Target population, geographic distribution, and type of questionnaire all affect costs.

4. Generalization needs: If you need a broad sample, you can’t do drop-off surveys. If you want to sample from the elderly, you should avoid electronic surveys as currently conceived.

Surveys So you have to think about the strengths

and weaknesses of your survey delivery method.

In a minute you’ll break into five groups to discuss strengths and weaknesses of:

1. Mailed2. Group Administered3. Face-to-Face Interviews4. Telephone5. Electronic

I’ll give you an example using the dropoff:

Surveys

Drop Off Strengths versus Weaknesses

Surveys—Drop Off Strengths

Convenient for respondents, like mail survey.

Potentially higher response rates than mailed survey because the researcher has a presence (if dropped off at residence) or organizational endorsement (if dropped off at an organization).

Weaknesses

Time. Because of the personal contact, this uses more researcher time.

Challenges to generalizability— Hard to get a broad

sample. Target population must be pretty specific because of the limited ability to make contact with different people.

Lack of control over distribution.

Lower response rates than face-to-face and telephone (possibly)

Inaccuracy—Behavioral problems in groups, “playful” responses.

Surveys So you have to think about the strengths

and weaknesses of your survey delivery method.

Break into Five (5) groups. 1. Mailed2. Group Administered3. Face-to-Face Interviews4. Telephone5. Electronic

Spend twenty (20) minutes generating strengths and weaknesses of your group’s survey delivery type.

Surveys

You must weigh the strengths and weaknesses of the survey delivery methods to choose the one that is appropriate and feasible.

The following slides consider the strengths and weaknesses of five delivery methods (the lists are not exhaustive).

Surveys

Mailed Strengths versus Weaknesses

Surveys—Mailed Strengths

Cost: The greatest cost in most businesses

is wages. Mailed surveys reduce labor compared with face-to-face or phone. However, human data entry is expensive—use scan forms.

Low cost for respondent when postage paid envelope is provided

Convenience for participants

Reduced Bias—no interviewer

Facilitates generalizability—Easy to reach more people over large area

Better Geographic (and demographic) targeting

Weaknesses

Motivation,People may not be motivated to respond or respond thoroughly when researcher is not present

Lower response rates, compared with face-to-face or group administered

Challenges to accurate response: Assumes physical skills Literacy language abilitiesThis likely biases results.

Interpretations of Questions can be inaccurate—bias from no interviewer to assist.

Who fills it out?

Surveys

Group Administered Strengths versus Weaknesses

Surveys—Group Administered Strengths

Higher response rate, especially if a captive audience such as school.

Permits complex questions—Greater specificity and variation in question types because there’s someone to answer questions and explain.

Good for educationally challenged and children:Someone is there to help.

Fast way to get sample.

Inexpensive, especially if you don’t hire many people. You get many cases at once. Data entry is still an issue.

Weaknesses

Challenges to generalizability—Hard to get a broad sample. Target population must be pretty specific.

Scheduling. Requires a block of time convenient for all.

Inaccuracy—behavioral problems in groups may distract respondents or even lead to “playful” responses

Privacy—setting may allow others to view subjects’ responses

Surveys

Face-to-Face Interviews Strengths versus Weaknesses

Surveys—Face-to-Face Strengths

Permits complex questions—Greater specificity and variation in question types because there’s someone to answer questions, explain, and probe.

Highest response rate. Personal appeal increases likelihood that respondents will participate.

Convenience—Allows rescheduling of visits (helps with response rate too).

Reduces disability bias—interviewer may assist respondents

CAPI:If using computer entry of responses, interviewing doubles as data entry.

Weaknesses

Cost. Payroll, driving, and so forth are expensive compared with other forms.

Increased Bias. The appearance and style of

the interviewer can bias participation or answers.

Imagine questions about drug use or sexuality.

Interaction between interviewer and interviewee can lower reliability

Oral format limits types of questions that you can ask, although visuals may be used.

Interviewer safety is of concern and fear can lead to falsified questionnaires.

Surveys

Telephone Interviews Strengths versus Weaknesses

Surveys—Telephone Strengths

Permits complex questions—Greater specificity and variation in question types because there’s someone to answer questions, explain, and probe. Not as good as face-to-face.

Higher response rate (maybe) Personal appeal increases likelihood that respondents will participate. Not as high as face-to-face.

Convenience—Allows rescheduling of contact (helps with response rate too).

Fast—surveys can be quickly completed

Facilitates generalizability—Easy to reach more people over large area, cheaper than face-to-face but more expensive than mailed survey

Environmentally friendly compared with other methods except electronic

Weaknesses

Cost. Payroll is still an issue, but is cheaper than face-to-face.

Number of questions is limited by low tolerance for being on the phone for long.

Biases: The linguistic style of the interviewer can

bias participation or answers. Imagine questions about drug use or

sexuality. If using area codes for sampling frame in an

era of cell phones. Type of household that answers the phone.

Who has a phone?

Limits on the types of questions that you can ask: relies on short-term memory and quick

recall visuals are not possible

Falling response rates because of American attitude toward phone. Americans hang up on strangers or screen

calls Phone interruption is an annoyance

Surveys

Electronic Surveys Strengths versus Weaknesses

Surveys—Electronic Strengths

Cheap. Depending on type, data are automatically entered by respondents

Ease of editing questionnaire and analyzing data

Fast—Distribution and response time is reduced.

Higher response rates (maybe) than other methods, sometimes.

Reduced bias—You get more candid responses. One researcher found: Number of persons having sex with same-sex individuals doubles compared with mail and face-to-face surveys.

Some complexity permitted. Interaction can be available, either through faqs, e-mail, or call to researchers.

Environmentally friendly compared with other methods except telephone

Respondents can view “tally” of responses

Can reach samples on a global scale

Convenient for respondents

Weaknesses

Selection Bias & Generalizability: Must have computer access, but can be

used to generalize to computer-access population.

Lacks complete sampling frame Email contact may be treated as SPAM

Construction of questionnaire can be challenging—often need web or computer expertise

Challenges to accurate response: Assumes physical skills Literacy and language abilities Instructions may get tedious and confuse

respondents Require electronic literacyThis likely biases results.

Glitches can kill participation for some

Long-term response rates may be no better than other survey forms.

Who fills it out?

The Web Culture encourages “hacking” and falsification of information

Surveys—Electronic Of the types of electronic survey, those that carry us

toward the population coverage of mail surveys and the response rates of the face-to-face survey will be judged the best.

Current forms ranked from worst to best:1. Public computers in high-traffic areas—as bad as man

on the street convenience samples.2. Web pages—sampling can be done via normal

methods, but often aren’t3. Distributed as E-mail

Hybrids will likely rule the electronic survey in the future.