Upload
iris-phelps
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
for the GEMP 3 student
Advantages and disadvantages of questionnaires Design of questionnaires Different types of questions used How to avoid common problems and pitfalls
Three main methods◦Observation◦Interviews◦Questionnaires
A study may employ more than one method of data collection.
Health and the eating habits of school children at school(Survey, questionnaire)
The average waiting time at clinics and the factors that influence the waiting time.
(Survey, on-site observation) Analysis of the causes of cor pulmonale in children in
Johannesburg..(Survey, secondary data)
The understanding of the Core PHC package among health workers employed by the city
(Survey, on-site observation)
An instrument (form) to ◦ collect answers to questions◦ collect factual data – gathers information or
measures
A series of written questions / items in a fixed, rational order
A well designed questionnaire: Gives appropriate data which answers your research
question Minimises potential sources of bias Will more likely be completed
• Facts (Demographics)• age, sex, post code, number of children, religious affiliations,
etc
• Attitudes“perceptions” “beliefs” –, belief in aliens, attitude towards
crime, etc.
• Knowledgedescription of homeopathy, good dietary practice,
• Behaviours• dental attendance, medical history, criminal record, drug
taking habits
What kind of questions can you ask?What kind of questions can you ask?
1. List variables Sources include patients/subjects (focus groups, key
informant interviews), clinical observation, theory or conceptual framework, prior research, and expert opinion.
2. Borrow from other instruments Save development effort (reinventing the wheel) Borrow reliability, validity, variance estimates Facilitate comparison with previous studies
3. Solicit input from colleagues and friends
Can reach a large number of people relatively easily and economically (especially postal/E-mail questionnaires)
Provide quantifiable answers Relatively easy to analyse
Provides only limited insight into problem– Limited response allowed by questions– Maybe not the right questions are asked
Varying response – Misunderstanding/misinterpretation
Need to get it right first time– Hard to chase after missing data
Self-administrated◦ On-site◦ By post ◦ E-mail/Internet
Interviewer-administrated◦ Face to face ◦ Telephone
Advantages: Cheap and easy to administer Preserves confidentiality Completed at respondent's convenience Administered in a standard manner No influence by interviewer
Disadvantages: Low response rate Questions can be misunderstood No control by interviewer Time loss
Advantages: – Interviewer can clarify unclear questions– Literacy is not required– Interviewer can collect more complex answers and
observations– Interviewer can minimize missing and inappropriate
responses– Interviewer can prevent respondent
from answering out of sequence– Clarification of ambiguity– Quick answers
Disadvantages: – Interviewer bias– Needs more resources– Only short questionnaires possible– Difficult for sensitive issues
Highly structured Collects the same types of information Standardised Allows quantitative and systematic analysis
Questionnaire design is integral part of the study, not an “add-on”!
Planning the study:
Decide on goals of study
Know the subject – literature, experts
Formulate a hypothesis Define information needed to test the hypothesis
Determine study population: Know the respondents
– Occupation – Special sensitivities– Education– Ethnic– Language
Design questions: – Content of the questions– Format of the questions– Presentation and layout– Coding schedule (if appropriate)
Pilot and refine questionnaire
Complete study protocol: Determine
– Cost – Time – Sample size– Response rate
Three parts:
1. Introduction
2. Questions
3. Concluding remarks
1. Identification of the survey or respondent– Who you are– Who you work for– How and where you can be contacted
2. Purpose of survey– What you are investigating/researching
3. Explanation of respondent selection– Where you obtained the respondent’s name
4. Request for participation/provide incentive– Incentives– Anonymity– Confidentiality
5. Screening of respondent
Encourage the respondent to answer all questions as truthfully as possible
Emphasise that there are no "right" or "wrong" answers
Try to make the research sound interesting and relevant.
Use a form of language, in terms of word and sentence length, which will be comprehensible to the respondent. – Fleisch Reading Ease Score (Grade 8 maximum)
Provide an estimate of how long it takes to complete the questionnaire (be honest)
Thank the respondent again for taking part
Remind them to check they have answered all questions
Ask them to return the questionnaire in the envelope provided
KISS keep it short and simple
◦ Length of questionnaire: shorter response rates
Appearance affects◦ Response rate◦ Data summarisation and analysis
◦ easy on the eye
• Number all items and pages• Put an identifying mark on all pages• Put study title in bold on first page• Print directions in bold• Remember the Unique Identifier
Question order – Easy difficult– General particular– Factual abstract– Starting questions
• Simple• With closed format • Relevant to main subject• Non-threatening• Neither demographic nor personal questions
– Be aware of ordering effects!
Group questions by◦ Topic/ response options
Don’t put most important item last
Questionnaire likely to be completed if – relevant – logical
Clear focus on research question ◦ Avoid sidetracking◦ Avoid unnecessary information
Demographic information
Contact information (if non-anonymised)
Ensure that the opening questions are clearly relevant to the purpose of the study
Consider putting factual questions at the end of the questionnaire
Personal or sensitive questions should appear later
Consider including some open questions
Length:– No longer than absolutely necessary (2–4 pages)
– Use branching questions and skip patterns to reduce length
– Avoid “deceptively short” questions with high respondent burden (complex tables, rank ordering, mental calculations, “check all that apply”)
Questionnaire can be pre-coded Quicker and easier data entry
Examples:◦ Male 1 Female 2 Don’t know 3
◦ Ill 1 Not ill 0 Don’t know 9
◦ Single 1 Married 2 Separated 3 ◦ Divorced 4 Don’t know 5
Precoding the Questionnaire
• Numbers are preferred for two reasons: • Numbers are easier and faster to keystroke
into a computer file• Computer tabulation programs are more
efficient when they process numbers
5. One topic per questionAvoid ‘double-barreled’ questions“Do black Africans and Coloureds suffer from discrimination”
6. Specify an appropriate time frameTypical/usual versus maximal/minimalDepends on salience of topic
7. Improving recallAided recall (memory cues, prompts)Bounded recall (time window)Records or diaries
8. Mutually exclusive and exhaustive response options
– Use of ‘other’ fields
9. Consider question polarity and sequence (survey as conversation)
– Avoid ‘yea-saying’ patterns and carryover effects, but maintain consistency when possible
10. Check for technical accuracy
Do you approve or disapprove of the recreational use of cannabis?
(Please tick the appropriate box)
Yes
No
Can’t answer question using answer categories provided Change to Approve/ Disapprove?
Can you assume respondent knows what “recreational” means?
Have you suffered from headaches or sickness recently?
(Please tick the appropriate box)
Yes
No
Question double barrelled
What does “recently” mean?
Would you rather not use a medicated shampoo?
(Please tick the appropriate box)
Yes
No
Avoid double negatives
If homeopathic remedies were available to buy in large chemists, would you buy them?
(Please tick the appropriate box)
Yes
No
Do people behave as they say they will?
Do you go to the toilet a lot?
Ambiguity
◦ How often do you get up at night to PU? (pass urine)
◦ Should IVDUs be treated in the community?
Avoid jargon/abbreviations/slang
Do you think that the food in the hotel made you sick?◦ Did the hotel staff seem unhygenic to you? ◦ Do you agree that the hospital staff were close to
exhaustion?
Avoid leading questions
What age are you? 16-20 20-25 25-30 35-40
Use simple language- “old’ instead of “age”Avoid not mutually exclusive options
Be precise Did you swim often in the pool? ◦ Yes ◦ No vs.How often did you swim in the pool? ◦ Once ◦ Twice ◦ Three times or more ◦ Not at all ◦ Don´t know
Be appropriate Are you a drunkard? ◦ Yes ◦ No vs.How often did you consume alcoholic beverages during your
holiday? ◦ Daily ◦ 2-6 times/week ◦ Once a week ◦ Less than once a week ◦ Don´t know
Ask for one information at a time:
– Did you go swimming in the pool and have cocktails at the bar during night or day?
– Yes �– No �
Remember option “don’t know” Mutually exclusive and exhaustive answer options Vertical order of answer options
Be objectiveDid you drink the strange pink drink? ◦ Yes ◦ No vs.Which beverage did you consume? ◦ Water ◦ Beer ◦ Wine ◦ Karkadé ◦ None of them ◦ Don´t know
Be simpleDid you smoke not less than a mean amount of 7 cigarettes/2
days from 1999 onwards? ◦ Yes ◦ No vs.Did you smoke more than 2 packs of cigarettes per week for the
last 5 years?◦ Yes ◦ No ◦ Don´t know
Visual analogue (thermometer) scales Frequency-weighted multi-symptom indices Multi-symptom checklists Grading or ranking
Likert Scales Assumes an attitude can be quantified on a linear scale Assumes respondent can discriminate at least five points on this scale The scale can vary according to the question asked
Eg “Homeopathic Medicine contains many techniques which should be incorporated into mainstream medicine”
Strongly Agree
Agree Unsure Disagree Strongly
Disagree
3. Rating scaleDid you use condoms during the past six months?
◦ With… Always Sometimes Seldomly Never
◦ Steady partner ◦ Casual partners ◦ Sex workers
4. Numerical rating scaleHow useful would you think that information on hepatitis A
from the travel agency would be? (please circle)
1 2 3 4 5 6 7Not at all useful Very useful
5. Scales for measuring attitude (Likert)Travel companies should not offer anymore hotels
linked to hepatitis AI strongly disagree I disagree quite a lot I disagree just a little I’m not sure about this I agree just a little I agree quite a lot I strongly agree
Hypothetical questions -- predictions Presuming/leading questions Sensitive questions
◦ Personal details, health, age, income Questions that rely on memory Questions that rely on prior knowledge Mutually exclusive responses
◦ e.g. age brackets: 30 – 40? Or 40 – 50? Lengthy or overly detailed questions
Casual approach◦ “Did you by any chance murder your wife?”
Numbered card approach◦ “Will you please read off the number of this card which corresponds to
what become of your wife?” The “Everybody” approach
◦ “As you know, many people have been killing their wives these days. Do you happen to have killed yours?”
The “Other People” approach◦ “Do you know any people who have murdered their wives?” Pause for
reply…”and how about yourself?”
Pilot with a similar group of people to your intended subjects
Highlights problems before starting– Effects of alternative wording– Overall impression on respondents and interviewers– Final polishing after several amendments
Take it yourself, and get a colleague to do the same
Get help with critiques from sample population as well as people experienced in the domain
Run pilot study using samples like target population (ideally 5 – 10%)– But don’t fold this data in with your real study
Iterate (but you knew that)
A well designed questionnaire: Will give appropriate data which allow to answer your
research question Will minimise potential sources of bias, thus increasing
the validity of the questionnaire Will much more likely be completed
Developing Questions“Shoulds” of Question Wording
• Question should be focused on a single issue or topic. No “double-barreled” questions.
• Question should be brief.• Question should be interpreted the same way by
all respondents; no ambiguity in word meaning.• Question should use respondent’s core
vocabulary. Keep wording simple.• Question should be a grammatically simple
sentence if possible.
Developing Questions“Should Nots” of Question Wording
• Question should not assume criteria that are not obvious.
• Question should not be beyond the respondent’s ability or experience; also, you do the math.
• Question should not use a specific example to represent a general case.
• Question should not ask the respondent to recall specifics when only generalities will be remembered.
• Question should not require the respondent to guess a generalization.
The Questionnaire Development Process
Any Questions?Any Questions?