Questionnaire Design, Measurement and Scaling · Chapter Objectives ... Understand the concepts of measurement and scaling. ... Measurement and Scaling Measurement

1- 1 Malhotra Hall Shaw Oppenheim Essentials of Marketing Research © Copyright 2004 Pearson Education Australia

PART THREE

Chapter 7

Questionnaire Design,

Measurement and Scaling

7-2 Malhotra Hall Shaw Oppenheim Essentials of Marketing Research © Copyright 2004 Pearson Education Australia

Chapter Objectives

After reading this chapter, you should be able to:

Explain the purpose of a questionnaire.

Describe the process of designing a questionnaire.

Understand the concepts of measurement and scaling.

Discuss the primary scales of measurement.

Classify and discuss scaling techniques as comparative and non-comparative.

Describe the various comparative and non-comparative scaling techniques.

Discuss the decisions involved in constructing itemised rating.


Questionnaires and Observation forms

Standardise data collection for survey and

observation, respectively.

Ensure comparability of the data, increase speed

and accuracy of recording and facilitate data

processing.

Definition

Formalised set of questions for obtaining

information from respondents


Objectives of a Questionnaire

Translate the information needed into a set of

specific questions that the respondents can and

will answer

To uplift, motivate, and encourage the

respondent to become involved in the interview,

to cooperate, and to complete the interview

Questionnaires should minimise response error


Figure 7.1 Questionnaire Design Process

Specify the information needed

Specify the type of interviewing method

Determine the content of individual questions

Design the questions to overcome the respondent’s inability and unwillingness to answer

Decide on the question structure

Determine the question wording

Arrange questions in proper order

Identify the form and layout

Reproduce the questionnaire

Eliminate bugs by pretesting


Specify the Information Needed

Consider the constructs

[theory, hypotheses/research question

developed earlier]

Keep in mind the target population and ask

questions appropriately

[tertiary educated may have different levels of

understanding compared with migrant groups]


Type of Interviewing Method

Personal interviews

Lengthy, complex and varied questions

Telephone interviews

Short and simple questions

Mail questionnaire

Simple questions, detailed instructions


Individual Question Content

Is the question necessary?

Every question should contribute to the

information needed or serve some

specific purpose.

[questions to develop rapport, questions which

disguise the purpose or sponsorship of the project,

questions may be duplicated for the purpose of

assessing reliability or validity]


Individual Question Content cont.

Are several questions needed instead of one?

Avoid ambiguity and double barrel questions.

Are you fit and healthy?

Do you prefer to patronise a department store

close to your home or work?


Overcoming Inability to answer

Is the respondent informed?

Filter questions are often used to ascertain

familiarity, product use and past experience

Are you the main grocery buyer in the household?

Have you purchased car tyres in the last 3 months?


Overcoming Inability to answer cont.

Can the respondent remember?

Inability to remember can lead to

Errors of omission

Telescoping

Creation

The ability to remember events are influenced by

the event itself

time elapsed since the event

presence/absence of other events that would

help memory


Overcoming Inability to answer cont.

Can the respondent articulate?

Respondents may be unable to describe the

situation accurately

Inability to articulate may lead to

respondents ignoring questions

refusing to respond to the rest of the

questionnaire


Overcoming Unwillingness to Answer

Effort required of the respondent

Unwilling to devote a lot of effort to provide

information

Context

Unwilling to respond to questions which are

considered inappropriate for the given context


Overcoming Unwillingness to Answer cont.

Legitimate Purpose

Unwilling to divulge information which they do

not see as serving a legitimate purpose

Sensitive information

Unwilling to disclose sensitive information

because this may cause embarrassment or

threaten the respondent’s prestige/ self-image


Increasing the willingness of respondents

Place sensitive topics at the end of the

questionnaire

Preface the question with a statement that the

behaviour of interest is common

Ask the question using the 3rd person technique

Provide response categories rather than asking

for specific figures

Use randomised techniques


Choosing Question Structure

Unstructured questions

Open-ended questions that respondents answer

in their own words

Express general attitude

Interviewer bias is high

Coding is costly and time consuming

What were you expecting by attending this event?


Choosing Question Structure cont.

Structured questions

Specify the set of response alternatives and the

response format

Types: multiple-choice, dichotomous, or a scale



Multiple-choice questions

Respondents are asked to select one or more alternative

Which of the following people had an influence on your

choice of university?

None

Parents

Friends

Ex-Uni student

Teachers at secondary school

Careers teachers at secondary school

Colleagues

Other please specify………………….



Dichotomous questions

Only two response alternatives

Have you stayed in Japan?

Yes No



Scale

Can be comparative or non-comparative

Do you intend to buy a new computer within the next 6

months?

Definitely Probably Undecided Probably Definitely

will not buy will not buy will buy will buy

1 2 3 4 5



About structured question

Can be administered quickly

Coding and processing is less costly and less

time consuming

Effort required in designing alternatives

May require exploratory research for determining

choices

Bias response because people pick alternatives

offered

Potential order bias


Choosing Question Wording

Define the issue

Define the issue by considering

6 Ws

Use ordinary words

Match the vocabulary level of the respondents

Avoid jargon and technical marketing words

Avoid ambiguous words

The word should have a single meaning which is

known to the respondents


Choosing Question Wording cont.

Avoid leading or biasing questions

Avoid clues which will lead the respondent to

answer in a particular way

Avoid implicit alternatives

Alternatives should be explicitly stated

Avoid implicit assumptions

Implicit assumptions about what will happen as a

consequence


Choosing Question Wording cont.

Avoid generalisations and estimates

Respondents should not have to make

generalisation, only specific questions should be

asked

Use dual statements – positive and negative

Avoid bias by altering statements


Determining the Order of Questions

Opening Questions

Important in gaining confidence and co-

operation of respondents

Questions should be interesting and non-

threatening

Could be qualifying question


Determining the Order of Questions cont.

Type of Information

Basic

Classification

Identification



Difficult questions

Sensitive, embarrassing, complex or dull

questions should be placed late in the sequence

Effect on subsequent question

Questions asked early in a sequence can

influence the responses to subsequent

questions



Logical order

All questions that deal with a particular

issue/topic should be asked before proceeding

to a new topic

Use branching questions to direct respondents

to other questions


Figure 7.2 Flowchart for questionnaire design


Form and Layout

Format, spacing and position are important

A questionnaire can be divided into several parts

All questions should be numbered

Questionnaires are sometimes pre-coded where

each possible response to a question is

associated with a unique number (or letter)

Questionnaire should be serially numbered


Reproduction of the Questionnaire

Use good quality paper and have a professional

appearance

Consider using booklet if the questionnaire runs

to several pages

Avoid splitting a question and response

categories over two pages

Use single columns for response categories

Avoid overcrowding questions

Directions or instructions for individual

questions should be placed close to the

questions as possible eg [PROBING]


Pre-testing

Testing the questionnaire on a small sample of

respondents to identify and eliminate potential

problems with [Question content, wording,

sequence, form and layout, question difficulty,

instruction]

Use protocol analysis or debriefing

After each significant revision of the

questionnaire, another pre-test should be

performed with a different sample

Responses obtained should be coded and

analyses


Observational forms

Specify the who, what when, where, why and way

of behaviour to be observed

Offers consistency, structure, completeness


Measurement and Scaling

Measurement

Assigning numbers (symbols) to certain

characteristics of objects according to certain

pre-specified rules

Scaling

Creating a continuum upon which measured

objects are located


Primary Scales of Measurement

N ominal

O rdinal

I nterval

R atio


Table 7.2 Primary Scales of Measurement

Scale Basic

Characteristics

Marketing

Examples

Descriptive

Statistics

Inferential

Statistics

No

min

al

Numbers identify

and classify

objects

Brand numbers,

store types,

gender

classification

Percentage,

mode

Chi-square,

binominal test

Ord

ina

l

Numbers indicate

relative position

not magnitude

Preference

rankings, market

position, social

class

Percentile,

median

Rank order

correlation,

Friedman

ANOVA

Inte

rval

Differences

between objects

can be compared

Attitudes,

opinions, index

numbers

Range,

mean,

standard

deviation

Correlations, t

tests, ANOVA,

regression,

factor analysis

Ratio

Zero point is fixed;

ratios of scale

values can be

computed

Age, income,

costs, sales,

market share

Geometric

mean,

harmonic

mean

Coefficient of

variation


Figure 7.3 A Classification of Scaling Techniques


Comparative Scaling Techniques

Paired comparison scaling

A respondent is presented with two objects and asked to select one accordingly to some criterion

Do you prefer Qantas or Virgin Blue?

Ordinal in nature

Analysis: percentages of respondents; transitivity of preference

Modifications: include a neutral / no opinion response; graded comparison where respondent is asked which brand is preferred and how much it is preferred


Comparative Scaling Techniques cont.

Rank order scaling

A respondent is presented with several objects simultaneously and asked to rank them

Place a 1 beside your most preferred soft drink; 2…

Diet Pepsi

Pepsi

AC Cola

Coke

Ordinal in nature

Used to measure preferences for brands and attributes



Constant sum scaling

A respondent is allocated a constant sum of units such as points among alternatives presented

Average is taken from all respondents

Ordinal in nature

Factors important when buying a car No of points

Style 20

Ride 10

Economical 25

Warranty 5

Price 40

Friendly Dealer 0

-------

100



Q-sort

Rank order procedure where objects are sorted

into piles based on similarity with respect to some

criterion

Number of objects between 60 and 90 can be

ranked

Pre-specified number in each class

Each object is than assigned a rank order


Non-Comparative Scaling Techniques

Continuous rating scale Respondents rate objects by placing a mark at the

appropriate position on a line that runs from one extreme of

the criterion variable to the other.

Interval data

Easy to construct but scoring is cumbersome and unreliable

To what extent do you involve your children in family decision making?

Not at all Extremely involved

1 2 3 4 5 6 7 8 9 10

Alternatively, mark with an X


Example: Rapid Analysis and Testing Environment

RATE is a continuous measurement of “gut reaction”.

Roy Morgan’s The Worm can measure response to

concept testing, advertising, even election debates.

Between 50 to 100 respondents are provided with a

Worming dial and asked to record their reaction to the

material being tested.

The information is fed into a computer which tabulates

second-by-second response profiles. The responses are

also stored in a data file for further analysis.

Respondent demographics are obtained via a

questionnaire and used to cross-tab the key information.

Source: www.roymorgan.com


Non-Comparative Scaling Techniques cont.

Itemised

rating

scales

Likert scales

Semantic

differential

scale

Stapel

scale



Likert Scale

Respondents indicate a degree of agreement or

disagreement [5 response alternatives] with each of a series

of statements about a stimulus object

Str

on

gly

dis

ag

ree

Dis

ag

ree

Ne

ith

er

ag

ree

no

r

dis

ag

ree

A

gre

e

Str

on

gly

Ag

ree

The website displays is visually pleasing design 1 2 3 4 5

Learning to operate the website is easy for me 1 2 3 4 5

The website is innovative 1 2 3 4 5

The website loads quickly 1 2 3 4 5

I trust the website to keep my personal information

safe

1 2 3 4 5

Shopping at this website is usually a satisfying

experience

1 2 3 4 5

I will continue to purchase from this Website. 1 2 3 4 5



Semantic differential Scale

Seven point rating scale with end points associated with bipolar

labels that have semantic meaning

Used in comparing brand, product and company images

Please evaluate the quality of Australian designed cars by placing an X

on the blank that best indicates your feelings

Unimaginative _ _ _ _ _ _ _ _ Creative

Unattractive _ _ _ _ _ _ _ _ Attractive

Obsolete _ _ _ _ _ _ _ _ State-of-the-art

Inefficient _ _ _ _ _ _ _ _ Efficient

Common _ _ _ _ _ _ _ _ Exclusive

Poor Performance _ _ _ _ _ _ _ _ Excellent Performance

Inferior Product design _ _ _ _ _ _ _ _ Superior Product Design



Stapel Scale

Unipolar rating scale with 10 categories without a neutral

point

Respondents indicate by selecting an appropriate numerical

response category how accurately or inaccurately each term

describes the objects

No need to pre-test adjectives

The website displays is visually pleasing design -5 -4 -3 -2 -1 +1 +2 +3 +4 +5

The website is innovative -5 -4 -3 -2 -1 +1 +2 +3 +4 +5

The website is creative -5 -4 -3 -2 -1 +1 +2 +3 +4 +5

The website loads quickly -5 -4 -3 -2 -1 +1 +2 +3 +4 +5

I feel safe in my transactions with the website -5 -4 -3 -2 -1 +1 +2 +3 +4 +5


Non-Comparative Itemised Rating Scale

Decisions

Number of scale categories

Balanced versus unbalanced scales

Odd or even number of categories

Forced versus non-forced scales

Nature and degree of verbal description

Physical form or configuration


Number of scale categories

5-point scale

The website is innovative

• Strongly disagree

• Disagree

• Neither agree nor disagree

• Agree

• Strongly Agree

7-point scale

The website is innovative

• Strongly disagree

• Disagree

• Disagree somewhat

• Neither agree nor disagree

• Agree somewhat

• Agree

• Strongly agree


Balanced versus unbalanced scales

Balanced

How important is price, as far as you are concerned?

• Very important

• Of some importance

• Of little importance

• Of absolutely no

importance

Unbalanced scales

How important is price, as far as you are concerned?

• Very important

• Of some importance

• Of little importance

• Of absolutely no

importance


Forced versus non-forced scales

Forced scales

In general how interested

are you in trying a new

brand of toothpaste?

• Very interested

• Somewhat interested

• Not too interested

• Not at all interested

Non-forced scales

In general how interested

are you in trying a new

brand of toothpaste?

• Very interested

• Somewhat interested

• Unsure

• Not too interested

• Not at all interested

Documents

Questionnaire Design, Measurement and Scaling · Chapter Objectives ... Understand the concepts of measurement and scaling. ... Measurement and Scaling Measurement