GSSR

Research Methodology and Methods of Social Inquiry

www.socialinquiry.wordpress.com

December 20, 2010

Survey Research (cont. De 13)

Research Using Available Data

II. Survey Research (continued Dec. 13)

Survey Modes• Questionnaires; • Interviews; • Mixed-mode Surveys

Questionnaires: - the respondent completes himself/herself & returns by a

specified deadline;

Mail questionnaire (pen-and-pencil; internet) Group-administered questionnaire Household drop-off surveys (home/business)

Interviews: - completed by the interviewer based on what the

respondent says;

Telephone, Internet, Face-to-Face, Focus Groups

CATI

CAPI

Mixed-mode Surveys:

- combination of modes to sample and/or collect the data.

Constructing the Survey

The Survey instrument should:

- ensure effective two-way communication between respondents and the researcher;

- assist respondents in recalling & clarifying their experiences, attitudes and thoughts;

- keep the respondent interested and motivated.

Types of Questions

A. Open-ended & closed-ended questions

B. Direct and Indirect questions

C. Funnel sequence questions

D. Reason Analysis

E. Filter or Contingency Questions

See www.socialresearchmethods.net/kb/questype.php

A. Open-ended questions:

How would you rate the President’s performance in office so far?

Closed-ended questions:

How would you rate the President’s performance in office so far? () Poor() Below average() Average() Above average() Excellent

See Singleton & Straits, p. 269-270 for considerations on when to choose open vs. closed-end questions.

B. Direct and Indirect questions

“Would you mind having a woman as supervisor?”

vs.

“Do you believe your co-workers would mind having a woman as supervisor?”

C. Funnel sequence (questions): - move from a very general question to progressively

more specific questions.

Inverted funnel sequence: begins with the most specific questions, and ends with the most general

D. Reason Analysis: - well-devised serious of questions to get at R’s decision

process, instead of asking just “Why?”

Understanding students’ choice of going to UW:

1. Decision to go to college “When did you first consider seriously going to college? How did you

reach this decision?

2. Selection of schools applied to“Did you apply to any other colleges and univ? Which ones?”

3. Final choice of UW

“Did your parents, friends, teachers or other persons help you come to this decision? Who? How much influence did this have on you?”

E. Filter or Contingency Questions

- to determine whether the respondent is qualified to answer your question of interest.

Response Formats

Structured & Unstructured

Structured

Dichotomous: the question has 2 possible responses;

Yes/No, True/False or Agree/Disagree response.

Questions using Rating Scales

Survey questions that attempt to measure on an interval level:

Likert response scale

Cumulative or Guttman scaleR check each item with which they agree. The items themselves

are constructed so that they are cumulative

The semantic differential rating approach: an object is assessed by the respondent on a set of bipolar adjective pairs (using 5-

point, or 7-point rating scale):

General Issues on Questions & their Content

Is the question necessary/useful?

Are several questions needed?

Does it cover all possibilities?

Double-barreled question (a question in which 2 separate ideas are represented together as a unit)

“What are your feelings towards African-Americans and Hispanic-Americans?”

- “and” “or”

Does the question need to be more specific?

“How well did you like the book?” on scale ranging from "Not At All" to "Extremely Well."

What does it mean to say you liked a book very well?

Instead:

“Did you recommend the book to others?”

or

“Did you look for other books by that author?”

Avoid Biased/emotionally loaded questions

Leading questions

- suggest a possible answer/ make some responses seem more acceptable than others

“How often do you smoke marijuana?”

(for certain underreported behavior, this type of questions may be necessary)

“Do you agree that ….”

Are the questions personal or sensitive (i.e. will respondents answer truthfully)?

Can the question be misunderstood?

What assumptions does the question make?

Specify the time frame

“Do you think the government will increase taxes?”

Research Using Available Data

(Secondary Analysis)

- use of already existing sources of data;

- generally refers to re-analysis of quantitative data;

However: Content analysis

Content Analysis

- analysis of text documents; - typically, the major purpose = identification of patterns in

text; - unobtrusive; - can be a relatively rapid method for analyzing large

amounts of text.

Involves:

- selecting and defining a set of content categories;- defining & then sampling the elements of the text that

are described by the categories;

- quantifying the categories (ex. frequency of occurrence)

- analyzing the data: relating content categories to one another, or to other variables.

Thematic analysis of text

The identification of themes or major ideas in a document or set of documents

(field notes, newspaper articles, technical papers, organizational memos, …)

Indexing

- wide variety of automated methods for rapidly indexing text documents.

Ex: Key Words in Context (KWIC) analysis = computer analysis of text data.

Key word = any term in the text that is not included in an exception dictionary (exception dictionary includes all non-essential words like "is", "and", and "of").

All key words are alphabetized & are listed with the text that precedes and follows it, so the researcher can see the word in the context in which it occurred in the text.

Quantitative descriptive analysis

- describe features of the text quantitatively

Basic systems of enumeration

1. Time/space measures; Ex: space devoted to certain topics in newspaper articles;

2. Appearance; when does a category appear in the text;

3. Frequency; Assumptions:

a) frequency of a category is a valid indicator of its importance, value, intensity;

b) each individual count is of equal importance, value or intensity;

4. Intensity-devise mechanisms to judge the intensity of the category

(similar to constructing scales and indexes).

Likely Methodological Problems

a) the types of information available in text form are limited

b) Bias due to sampling

c) Bias due to (mis)interpreting results of automated content analyses