View
228
Download
0
Category
Preview:
Citation preview
Designing the QuestionnaireDesigning the Questionnaire
Questionnaire building Questionnaire building is an art!is an art!
A questionnaireis always custom-built!
The process of questionnaire designThe process of questionnaire designPLANNING WHAT TO MEASURE
Revisit the research objectives
Decide on the research issue of your questionnaire
Get additional information on the research issue from secondary data sources and exploratory research
Decide on what is to be asked under the research issue
FORMATTING THE QUESTIONNAIRE
In each issue, determine the content of each question.
Decide on the format of each question
QUESTION WORDING
Determine how the question is worded
Evaluate each research question on the basis of comprehensibility, knowledge and ability, willingness/inclination of a typical respondent to
answer the question
FORMATTING THE QUESTIONNAIRE
In each issue, determine the content of each question.
Decide on the format of each question
The process of questionnaire designThe process of questionnaire design
SEQUENCING AND LAYOUT DECISIONS
Lay out the questions in a proper sequence
Group all the questions in each subtopic to get a single questionnaire
PRETESTING AND CORRECTING PROBLEMS
Read through the whole questionnaire to check whether it makes sense and it measures what it is supposed to measure
Check the questionnaire for error
Pretest the questionnaire
Correct the problems
Designing the QuestionnaireDesigning the Questionnaire
Logical Steps to Develop a Good QuestionnaireLogical Steps to Develop a Good Questionnaire Plan what to measurePlan what to measure Formulate questions to obtain the needed informationFormulate questions to obtain the needed information Decide on the order and wording of questions and the Decide on the order and wording of questions and the
layout of the questionnairelayout of the questionnaire Using a sample, test the questionnaire for omissions and Using a sample, test the questionnaire for omissions and
ambiguityambiguity Correct the problems (pretest again, if necessary)Correct the problems (pretest again, if necessary)
Designing the Questionnaire (Contd.)Designing the Questionnaire (Contd.)
Planning What to MeasurePlanning What to Measure Specify research objectives and information to Specify research objectives and information to
be collected be collected Relevance of the studyRelevance of the study
Translating Research Objectives Into Translating Research Objectives Into Information RequirementsInformation Requirements
Designing the Questionnaire (Contd.)Designing the Questionnaire (Contd.)
Formatting the QuestionFormatting the QuestionDecision to be made regarding the degree of freedom to be Decision to be made regarding the degree of freedom to be given to the respondents in answering the questionsgiven to the respondents in answering the questions
AlternativesAlternatives Open ended with no classificationOpen ended with no classification Open ended where the interviewer uses precoded Open ended where the interviewer uses precoded
classifications to record the responseclassifications to record the response Close ended or structured format in which a Close ended or structured format in which a
question or a supplementary card presents the question or a supplementary card presents the responses to be consideredresponses to be considered
Open Ended QuestionsOpen Ended Questions Open Ended Questions Are Good for the Following Open Ended Questions Are Good for the Following
CircumstancesCircumstances Introduction to a survey or to a topicIntroduction to a survey or to a topic When it is important to measure the saliency of an issue to a When it is important to measure the saliency of an issue to a
respondentrespondent When there are too many responses to be listed, or they can't When there are too many responses to be listed, or they can't
be foreseenbe foreseen When verbatim responses are desired to give the flavor of When verbatim responses are desired to give the flavor of
people's answers or to cite examplespeople's answers or to cite examples When the behavior to be measured is sensitive or disapprovedWhen the behavior to be measured is sensitive or disapproved
Open Ended Questions (Contd.)Open Ended Questions (Contd.)
AdvantagesAdvantages Wide range of responsesWide range of responses Responses obtained without any influenceResponses obtained without any influence Free choicesFree choices
Open Ended Questions (Contd.)Open Ended Questions (Contd.)
DisadvantagesDisadvantages Variability in the clarity and depth of the responses Variability in the clarity and depth of the responses Articulateness of the respondent in personal Articulateness of the respondent in personal
interviewinterview Willingness to compose a written answer for a Willingness to compose a written answer for a
mail surveymail survey Interviewer's ability to record the verbatim answers Interviewer's ability to record the verbatim answers
quicklyquickly
Open Ended Questions (Contd.)Open Ended Questions (Contd.)
Disadvantages (Cont.)Disadvantages (Cont.) These type of questions are also time consumingThese type of questions are also time consuming Involves subjective judgements during summarization Involves subjective judgements during summarization
and are prone to errorand are prone to error ExpensiveExpensive The answer to this type of question expands or contracts The answer to this type of question expands or contracts
depending on the space or time available for itdepending on the space or time available for it Respondents may not use the same frame of reference, Respondents may not use the same frame of reference,
when the options are not availablewhen the options are not available
Closed-response QuestionsClosed-response Questions
There Are Two Basic Formats for Closed Ended There Are Two Basic Formats for Closed Ended or Structured Questionsor Structured Questions
Choice from a list of responsesChoice from a list of responses Appropriate single-choice rating on a scaleAppropriate single-choice rating on a scale
Closed-response Questions (Contd.)Closed-response Questions (Contd.)
AdvantagesAdvantages Easier to answerEasier to answer Require less effort by the interviewerRequire less effort by the interviewer Tabulation and analysis is easierTabulation and analysis is easier Less potential error in the way the question is Less potential error in the way the question is
asked and the way it is recordedasked and the way it is recorded The responses are directly comparable from The responses are directly comparable from
respondent to respondentrespondent to respondent
Closed-response Questions (Contd.)Closed-response Questions (Contd.)
Limitations Limitations Disagreement among researchers on the type of Disagreement among researchers on the type of
responses that should be listedresponses that should be listed The answer to a closed response question will be The answer to a closed response question will be
received no matter how relevant or irrelevant the received no matter how relevant or irrelevant the question is in that contextquestion is in that context
May not produce meaningful resultsMay not produce meaningful results Dichotomous questions are prone to a large amount of Dichotomous questions are prone to a large amount of
measurement error because the alternatives are polarizedmeasurement error because the alternatives are polarized
Closed-response Questions (Contd.)Closed-response Questions (Contd.)
Limitations (Contd.)Limitations (Contd.) Good questions are hard to developGood questions are hard to develop Provides fewer opportunities of self expressionProvides fewer opportunities of self expression The list of alternative responses provides The list of alternative responses provides
answers that might have not been considered by answers that might have not been considered by the respondent who are reluctant to admit the respondent who are reluctant to admit ignorance, and thereby selecting a "reasonable" ignorance, and thereby selecting a "reasonable" responseresponse
Number of Response CategoriesNumber of Response Categories
Generally five to seven categoriesGenerally five to seven categories Ideally the multiple choices should be Ideally the multiple choices should be
mutually exclusivemutually exclusive
Order of Response CategoriesOrder of Response Categories
Responses are likely to be affected by the Responses are likely to be affected by the order in which they are presented.order in which they are presented.
What factor influences your fast-food What factor influences your fast-food restaurant choice most ?restaurant choice most ?
Convenient locationConvenient location Quality of foodQuality of food Menu selectionMenu selection Fast serviceFast service Reasonable pricesReasonable prices Brand nameBrand name CleanlinessCleanliness
Order of Response CategoriesOrder of Response Categories To prevent order bias, place the average or To prevent order bias, place the average or
expected response at various positions in the expected response at various positions in the sequence of categoriessequence of categories
Range of Response CategoriesRange of Response Categories Respondents who do not know the answer Respondents who do not know the answer
might take categories as cues.might take categories as cues.
How many long-distance calls do make in a How many long-distance calls do make in a week?week?
less than 5less than 5 less than 10 less than 10 5-105-10 or or 10-20 10-20 More than 10.More than 10. More than 20. More than 20.
Handling Uncertainty and IgnoranceHandling Uncertainty and Ignorance
Concerns the handling of “don’t know” and Concerns the handling of “don’t know” and neutral responsesneutral responses
May be advisable to provide the interviewer May be advisable to provide the interviewer with an additional “no answer” category to with an additional “no answer” category to identify these people correctlyidentify these people correctly
Using Both Open-Response And Closed-Using Both Open-Response And Closed-Response QuestionsResponse Questions
Probe:Probe:Using an open-response question to follow up a Using an open-response question to follow up a closed-response questionclosed-response question
Two general purposes for the use of probes:Two general purposes for the use of probes: Pinpoint questions that were particularly difficult Pinpoint questions that were particularly difficult
for respondentsfor respondents Aid researcher interpretation of respondent Aid researcher interpretation of respondent
answersanswers
Question WordingQuestion WordingCare Has to Be Taken ThatCare Has to Be Taken That Is the vocabulary simple, direct, and familiar to all Is the vocabulary simple, direct, and familiar to all
respondents?respondents? Do any words have vague or ambiguous meanings?Do any words have vague or ambiguous meanings? Are any questions " double-barreled”?Are any questions " double-barreled”? Are any questions leading or loaded?Are any questions leading or loaded? Are the instructions potentially confusing?Are the instructions potentially confusing? Is the question applicable to all respondents?Is the question applicable to all respondents? Are the questions of appropriate length?Are the questions of appropriate length?
Question WordingQuestion Wording Avoid ambiguous wordsAvoid ambiguous words
How many times per month do you How many times per month do you visit a fast-food restaurant?visit a fast-food restaurant?
NeverNever OccasionallyOccasionally SometimesSometimes OftenOften
Question WordingQuestion Wording Are any questions loaded?Are any questions loaded?
1) Don’t you think, because its so greasy, 1) Don’t you think, because its so greasy, fast-food is one of the worst types of fast-food is one of the worst types of food? food?
2) Do you prefer a hamburger that is 2) Do you prefer a hamburger that is grilled on a hot stainless-steel grill or grilled on a hot stainless-steel grill or cooked by passing the raw meat through cooked by passing the raw meat through an open gas flame?an open gas flame?
Question WordingQuestion Wording Are any questions "double-barreled”?Are any questions "double-barreled”?
Are you satisfied with the price and the Are you satisfied with the price and the service of Taco Bell?service of Taco Bell?
Question WordingQuestion Wording Is the question applicable to all respondents?Is the question applicable to all respondents?
Why do you like fast-food?Why do you like fast-food?
Assumes that respondent likes fast-food.Assumes that respondent likes fast-food.
Asking Sensitive QuestionsAsking Sensitive Questions
Consumption of Kellogg’s Frosted FlakesConsumption of Kellogg’s Frosted FlakesThe casual approachThe casual approach
““Have you eaten ‘Frosted Flakes’ within the last week?Have you eaten ‘Frosted Flakes’ within the last week?
The numbered cardThe numbered card ““Would you please read off the number on this card that Would you please read off the number on this card that
corresponds to what you had eaten for breakfast in the last corresponds to what you had eaten for breakfast in the last week?”week?”
(Hand card to respondent)(Hand card to respondent)1.1. PancakesPancakes2.2. Frosted FlakesFrosted Flakes3.3. Other (what)?Other (what)?
Asking Sensitive Questions (Cont.)Asking Sensitive Questions (Cont.)
The everybody approachThe everybody approach ““As you know, many people have been eating As you know, many people have been eating
Frosted Flakes for breakfast. Do you eat Frosted Frosted Flakes for breakfast. Do you eat Frosted Flakes?”Flakes?”
The “other people” approachThe “other people” approach ““Do you know of any adult who eats Frosted Do you know of any adult who eats Frosted
Flakes?”Flakes?” ““How about yourself?”How about yourself?”
Asking Sensitive Questions (Cont.)Asking Sensitive Questions (Cont.)
The sealed ballot techniqueThe sealed ballot technique In this version you explain that the survey respects people’s In this version you explain that the survey respects people’s
right to anonymity with respect to their eating habits, and that right to anonymity with respect to their eating habits, and that they themselves are to fill out the answer to the question, seal they themselves are to fill out the answer to the question, seal it in an envelope, and drop it in a box conspicuously labeled it in an envelope, and drop it in a box conspicuously labeled “sealed ballet box” that is carried by the interviewer“sealed ballet box” that is carried by the interviewer
The Kinsey approachThe Kinsey approach Stare firmly into respondent’s eyes and ask in simple clear-cut Stare firmly into respondent’s eyes and ask in simple clear-cut
language as that to which the respondent is accustomed, and language as that to which the respondent is accustomed, and with an air of assuming that everyone had done everything, with an air of assuming that everyone had done everything, “Do you eat Frosted Flakes for breakfast?”“Do you eat Frosted Flakes for breakfast?”
Asking Sensitive Questions (Cont.)Asking Sensitive Questions (Cont.)
Randomized Response TechniqueRandomized Response Technique The respondent is asked to answer one or two randomly The respondent is asked to answer one or two randomly
selected questions without revealing which question has been selected questions without revealing which question has been answeredanswered
Questions:Questions: SensitiveSensitive InnocuousInnocuous
Since the interviewer records a “yes” or “no” answer Since the interviewer records a “yes” or “no” answer without knowing which question has been answered, without knowing which question has been answered, the respondent feels free to answer honestlythe respondent feels free to answer honestly
Randomized Response TechniqueRandomized Response Technique
P[Yes] = P[Yes|P[Yes] = P[Yes|S.QS.Q] * P[S.Q] + P[Yes|] * P[S.Q] + P[Yes|I.QI.Q] * ] * P[I.Q]P[I.Q]
wherewhere
S.Q = Sensitive QuestionS.Q = Sensitive Question
I.Q = Innocent QuestionI.Q = Innocent Question
Sequence And Layout DecisionsSequence And Layout Decisions
Open with an easy and non threatening Open with an easy and non threatening questionquestion
The questionnaire should flow smoothly and The questionnaire should flow smoothly and logically from one topic to the nextlogically from one topic to the next
Proceed from broad general questions to the Proceed from broad general questions to the more specificmore specific
Organization of a Typical QuestionnaireOrganization of a Typical QuestionnaireLocation Type Function Example
Starting questions Broad, generalquestions
To break the ice andestablish a report withthe respondent
Do you own apersonal computer?
Next few questions Simple and directquestions
To reassure therespondent that thesurvey is simple andeasy to answer
What brands ofpersonal computersdid you considerwhen you bought it?
Questions up to athird of thequestionnaire
Focused questions Relate more to theresearch objectivesand convey to therespondent the area ofresearch
What attributes didyou consider whenyou purchased youpersonal computer?
Major portion of thequestionnaire
Focused questions;some may be difficultand complicated
To obtain most of theinformation requiredfor the research
Rank the followingattributes of apersonal computerbased on theirimportance to you
Last few questions Personal questionsthat may be perceivedby the respondent assensitive
To get classificationand demographicinformation about therespondent
What is the highestlevel of educationyou have attained?
Sequence And Layout Decisions Sequence And Layout Decisions (Contd.)(Contd.)
Sensitive questions should not be placed in the Sensitive questions should not be placed in the beginning of the questionnairebeginning of the questionnaire
Use good quality of paperUse good quality of paper Physical layout should be appealing and Physical layout should be appealing and
interestinginteresting Order biasOrder bias
Order Bias: Does The Question Create Order Bias: Does The Question Create The Answer?The Answer?
1. No question asked1. No question asked 2.82.8
2. Asked only about advantages2. Asked only about advantages 16.716.7
3. Asked only about disadvantages3. Asked only about disadvantages 0.00.0
4. Asked about both advantages and 4. Asked about both advantages and disadvantagesdisadvantages
5.75.7
Percentage of Respondents “Very Much Interested” in Buying New ProductQuestions Preceding Buying Interest Question
Pretesting and Correcting ProblemsPretesting and Correcting Problems Pretest DesignPretest Design Pretesting Specific Questions ForPretesting Specific Questions For
VariationVariation MeaningMeaning Task difficultyTask difficulty Respondent interest and attentionRespondent interest and attention
Pretesting the QuestionnairePretesting the Questionnaire Flow of the questionnaireFlow of the questionnaire Skip patternsSkip patterns LengthLength Respondent Interest and AttentionRespondent Interest and Attention
ExercisesExercises
Why did you purchase a Sony brand Why did you purchase a Sony brand Walkman?Walkman?
Price was lower than other alternativesPrice was lower than other alternatives Feel it represents the highest qualityFeel it represents the highest quality Availability of local serviceAvailability of local service Sound is betterSound is better Warranty is betterWarranty is better Looks goodLooks good
ExercisesExercises
Have you ever listened to a Long John Have you ever listened to a Long John Silvers radio commercial?Silvers radio commercial?
Which of the following restaurants do you Which of the following restaurants do you visit frequently?visit frequently?
Burger KingBurger King Pizza HutPizza Hut James Coney IslandJames Coney Island
ExercisesExercises
Do you agree that, since fast-food Do you agree that, since fast-food restaurants produce a disproportionate restaurants produce a disproportionate amount of waste, they should be subject amount of waste, they should be subject to an additional environmental tax?to an additional environmental tax?
YesYes NoNo
ExercisesExercises
How often do you eat fast food?How often do you eat fast food?
DailyDaily Every second dayEvery second day Once a weekOnce a week Every two weeksEvery two weeks
Recommended