Questionnaire Design

Ed Blair

University of Houston

Broad issues in writing questions:

Have you asked the right question? Do respondents understand the question? Do they know the answer? Are they willing to answer accurately?

Have you asked the right question?In marketing research, we ask questions because the answers will help improve

marketing decisions. A question can be technically perfect, yet still be a bad question because it doesn't properly serve our marketing purpose.

Is this a good question?

1. Have you heard of any physician referral services that you can call if you need help in finding a doctor?

Yes . . . . . . 1 (ASK Q1A)

No . . . . . . 2 (GO TO Q2)

(IF YES)

1A. Can you name any of these services or tell me their telephone numbers? (PROBE: Any others?)

Making sure it’s the right question

Ask yourself: Is this question consistent with my theory about

how the market works? Do I know, specifically, how I plan to use the

answers to this question to draw conclusions about the market?

Do R’s understand the question? Do respondents understand the question?

Q: Does your home have power conditioning equipment to protect your critical equipment against power fluctuations?

Q: (en Español?)

Do all respondents interpret the question the same way?

Q: What is your income?

Do respondents interpret the question the same way you do?

Q: What are all the reasons you bought your groceries at ValueLand rather than some other store? (PROBE: Any others?)

Minimizing problems from misunderstanding

Ask questions one at a time.BAD Do you consider a backup generator or a surge suppressor to be necessary equipment for your home?

Use simple, specific language, and avoid words with more than one meaning.BAD? Do you exercise regularly?

Specify who, what, when, where, how, and the desired form of response.Which of the following categories best describes your family’s total income last year before taxes?

Before doing the project, have someone read you the questions. Also consider pre-testing your questionnaire to see whether respondents have trouble understanding some of the questions, and how they interpret key questions.

Do R’s know the answer? Were they there?

Q: How satisfied were you with the service you received from the hospital's business office?

Q: Why did your company…?

Can they remember?Q: In the past thirty days, about how many telephone calls did you make?

Do they have an opinion?Q: I have a list of hospitals in our area. As I read each one, please tell me how you would

rate that hospital as a place to receive care: excellent, good, fair, or poor. How about . . .

Are intentions meaningful?Q: If there was a local magazine like Consumer Reports that evaluated restaurants, auto

repair shops, stores, plumbers, and other local businesses, would you subscribe?

Q: How many small batteries do you intend to buy during the next six months?

Minimizing knowledge problems (a) Consider qualifying respondents for knowledge or allowing for “no opinion” or “not

sure” or “don’t know.”

Use the right time frame for your question.Event Reasonable time frameHospital stay, car purchase One year

Appliance purchase Three months

Grocery purchase One week

Food consumption One day

Measure behavior over intentions/attitudes (e.g., previous battery purchases over intent to purchase, actual referrals over intent to refer/satisfaction).

Conjoint, anyone?

Minimizing knowledge problems (b) Anchor subjective measures with comparisons across time, objects, or people.

78% of patients say they were "very satisfied" overall with the food at your hospital. Is this good or bad?

- What if the number was 84% last year?

- What if the number is 84% at another hospital?

- What if the number is 90% for maternity patients, 71% for oncology patients?

Use previous questions wherever possible!

Interpret and present self-report data as if they are approximate, not exact, and focus on patterns in the data rather than specific tabulations.

Are R’s willing to answer accurately?

Is the behavior or attitude socially desirable or undesirable?Q: During the past month, have you read any books other than for work or school?

Q: During the past month, since (DATE) , did you drink beer even once?(IF YES)

A. About how often did you drink beer during the past month?Once . . . . . . . . . . . . . . ……….1

2-3 times . . . . . . . . . . . ……....2

About once a week . . . . . . … .3

2-3 times a week . . . . . . . ……4

4-6 times a week . . . . . . . . …..5

Once a day or more . . . . . . . …6

Do respondents think you care which answer they give?Q: If there was a local magazine like Consumer Reports that evaluated restaurants, auto

repair shops, stores, plumbers, and other local businesses, would your family subscribe to this magazine?

Minimizing willingness problems (a) Professionalism is a must, to convince respondents that you care whether they

answer but not how they answer. This means: - Professional appearance for questionnaires that respondents will see.

- A proper and professional introduction.

- Formal language in the questionnaire (no slang, please)

- Business clothing for interviewers if respondents will see them

- Proper training so that interviewers seem to know what they are doing

Use choice measures or previous behavior rather than intention measures.

Check for loading. Don't ask subjective questions (attitudes, opinions, etc.) in in a "Yes-No" format, and avoid “angel" or "devil" words (e.g., reasonable, forbid), absolute terms (always, never), and appeals to the norm (“most people think")

Minimizing willingness problems (b) Don't reveal sponsor identity unless necessary to convince respondents that their

answers will have a legitimate use.

Consider less personal methods (mail, online) for sensitive topics.

Use open questions for sensitive quantitative questions (except income)

Use longer questions to reduce the social stigma attached to certain answers.Q: Many people say that they just don't have time for reading anymore. In the past month,

have you read any books other than for work or school?

What’s wrong with this questionnaire? (a)

(Fan survey to be administered at an Astros game)

1. In what city do you live? ____________________

(IF HOUSTON)

A. In what part of Houston do you live?

Inside the Loop South

North Southwest

Northeast West

East Northwest

Southeast

2. How many people are in your group at tonight's game?

1 2-3 4-5 6 or more

What’s wrong with this questionnaire? (b)

3. How many of them are kids?

1 2-3 4-5 6 or more

4. How many people in your group, including yourself, are Astros season ticket holders?

1 2-3 4-5 6 or more

5. How many games have you, yourself attended this season?

1 2-3 4-5 6 or more

6. What four radio stations do you listen to most often?

__________________ _________________

__________________ _________________

What’s wrong with this questionnaire? (c)

7. Why did you come to tonight's Astros game?

__________________________________________

__________________________________________

__________________________________________

We need your name and address for our prize drawing.

NAME: ________________________________________

ADDRESS: ________________________________________

CITY, STATE, ZIP ___________________________________

Revised questionnaire (a)1. Are you attending tonight's game on a season ticket or a single game ticket?

Season ticket Single game ticket

2. Counting tonight's game, how many Astros games have you attended this season?

1 2-3 4-5 6 or more

3. Counting yourself, how many people are in your group at tonight's game?

1 2-3 4-5 6 or more

4. How many of the people in your group are less than 13 years of age?

0 1 2-3 4-5 6 or more

Revised questionnaire (b)5. When did you decide to attend tonight's game? Was it today, before today but

during the past week, or earlier than the past week?

Today Past week Earlier

6. What promotions, if any, might influence you to attend an Astros game?

(CHECK ALL THAT APPLY)

$2 off tickets Prize drawings

Family discounts Country music concerts

Free parking Rock music concert

Free Astros mug Tailgate parties

Free kid's cap Home run derby

Revised questionnaire (c)7. What is your favorite radio station?

(FILL IN) _____________________

We need your name and address for our prize drawing. │

NAME: ________________________________________

ADDRESS: ________________________________________

CITY, STATE, ZIP ____________________________________