Embed Size (px)
Citation preview
Survey Research Considerations
Sherry Larkins, Ph.D.University of California, Los Angeles
Integrated Substance Abuse ProgramsOctober 16, 2020
A question or problem arises
The question is refined into a hypothesis; variables selected
A sample is selected from the populationProcedures are
implemented
Systematic observations/
measurements are made; data
collected*
Analysis of observations/
measurements
Interpretation of results
Dissemination of findings
The Scientific Process
* Survey, Experiments, Field work, etc.
Steps in the Survey Research Process
Pilot Instrument
Target Population; Sample Selection
Conduct Surveys; Record Data
Present/Publish Findings
Develop hypotheses; Develop survey
Analyze Data
Surveys Most widely used data-gathering technique in Social
Sciences
Collects information from a SAMPLE of individuals through their responses to standardized questions –AKA “respondents”
Good for testing hypotheses generated through deductive process
Often uses random sampling approach, valid and reliable instruments, and statistical analysis of data
Most efficient way to describe large populations, tracking trends, self-reported beliefs, attitudes, opinions, behaviors
Surveys, cont. Surveys cover many things at one time; measure many
variables, & test several hypotheses within one survey
Mostly use CLOSED-Ended questions, not OPEN-ended(response choices listed)
Good for “what”, “where”, and “when” questions; Not good for “why” or “how” questions
Some develop their own surveys (“primary data collection”), and some more use existing data sources (“secondary data sources”)
Strong on reliability, measurement validity Uniformity and standardization: questions and response
choices are the same for everyone
Surveys, cont. General categories of survey questions (not
mutually exclusive) Behavior (How often do you use condoms? How
many sexual partners have you had?; When did you last vote in an election?)
Attitudes/beliefs/opinions – Do you approve of the president’s performance? How do you feel about euthanasia?
Characteristics/Description – What is your marital status? What is your age? Self-Classification – Do you consider yourself religious or not religious? To which social class do you belong?
Knowledge – How is HIV transmitted?
Survey Research Considerations
SURVEY CONTENT: What do we ask and how do
we ask it?
Survey Research
SURVEYS: Examples
Constructing Questionnaire Items Start with items validated in previous research Keep it clear, simple, with the respondent’s perspective in mind:
consider population’s cultural background, literacy level, etc. Consider skip patterns; (pre)screen Qs More of an art than a science Short items are best Avoid jargon/slang Avoid ambiguity or vagueness (ex. Do you exercise regularly?) Avoid emotional language; use neutral words (ex. Steal, murderous) Avoid prestige bias (ex. Titles, positions; “Doctor’s think….”) Avoid “double-barreled” questions: Items that contain more than one
concept Avoid Leading/loaded items (ex. you don’t view pornography, do you?) Avoid double-negative items (e.g., “who should be prohibited from…..;”
“The US should not recognize Cuba…”) Avoid Biased items (pro-life vs. pro-abortion; welfare vs. assistance to
poor) Avoid monotony….response set Be mindful or order effects (religion, etc), priming effects…..
Constructing Questionnaire Items, cont.
Do you disagree with those who believe in the Death Penalty?
Do you eat fast food often? Do you support providing health benefits to
those who are unemployed and poor? In the past month, how many times did you party
with your friends? How did you find our hotel service: Excellent,
Good, Fair Did you do your patriotic duty and vote in the last
election?
Questionnaire Content & Layout Include Instructions Avoid long questionnaires, small print, too much content Skip or contingency Qs? (Screening) Item order can affect responses (opening, middle,
ending questions) Moving from General to Specific (ex. Sexual behavior;
drinking and driving) Threatening – sensitive topics = under/over reporting;
ego protection; shame and embarrassment. Socially Desirable – Distorting answers to conform to
social norms; over-reporting “high-culture” behaviors –reading literature, donating$
Knowledge – pilot difficulty level; Pre-test the questionnaire
Single Item Vs. Composite Measures
Composite Measures – Measure that Combines multiple items to create - a single value Scale – composite measure that averages
responses to a series of related items that capture a single item or trait – measures intensity (agree vs disagree)
Index – ACES; An Index sums responses to survey items capturing key elements – single score
TYPES OF SURVEYS: Modes of Administration and
Conducting a Survey
Questionnaire Administration Self-Administered
Mailed surveys Internet surveys (sampling issues) Computer Assisted (CASI)
Interviews In-person Telephone
Should I Use Self-Administered Surveys or Interviews?
Self-Administered Less costly but feasible. Sensitive Topics Offer anonymity and avoid interviewer bias Low response rate No control over completion process (who fills it out) Web-based surveys easiest, but worst response rate; verification and
design issues
Interviews: Face to Face Less missing data, higher response rates; can get a lot of data Better for complicated layout or sampling method (household surveys) In-person interviewers can make observations CASI/CAPI - Computer-assisted via phone and in-person; (ACASI) Most costly b/c training required to ask face-to-face Qs Appearance, tone, and dynamic between interviewer and interviewee,
may interfere w/ responses (hot girl/guy) Interviewer effect – mere presence of interviewer alters response
(Vietnam; SBIRT project; interviewer vs. tablet collection); Social Desirabilty Bias = under vs. over reporting (male/female SPs)
InterviewsManaging interviewers
Interviewer can bias responses; goal is to be non-judgmental and not reveal feelings thoughts
Reliability – different interviewers may not obtain same responses from the same people
Less control over the way questions are askedCoordination and Control –important for
reliability
Interviews, cont. Training interviewers
Follow question wording exactly Probe for responses Record responses verbatim; Audio record with
permission
Supervising interviewers Important to examine completed interviews for
systematic problems, differences Regular meetings with interviewers to discuss
problem situations and how to handle them
Telephone Surveys
Representativeness of sample Most households have phones People screen their calls
Use random-digit dialing and CATI (Computer assisted telephone interviewing)
More control, easier to monitor/supervise interviewers
Survey Strengths/Weaknesses
Useful in describing large pops. (when combined with good sampling)
They make large studies feasible Measurement is cleaner; uniform; standardized But, standardization may mean fitting square
peg into round hole Rarely capture the whole of social life or context Self-report = some artificiality May be strong on reliability but can be weak on
validity
