Survey Instruments and Questionnaire Design

SURVEY INSTRUMENTS & QUESTIONNAIRE DESIGN

Warren Seuradge Strategic Planning, Policy, and Partnerships

Region of Peel (2014)

Step 1

Problem Definition and Research Objectives

Step 2

RESEARCH AND QUESTIONNAIRE DESIGN

Step 3

Data Analysis and

Measurement

!" S#rv"$ Pr%&"''

Step 1


Step 2


Step 3

Data Analysis and

Measurement

!" S#rv"$ Pr%&"''

Step 1


Step 1


Pr%b(") R"&%*+,-,%+

Step 1


Pr%b(") R"&%*+,-,%+

(1) Identify Source of the Problem

Step 1


Pr%b(") R"&%*+,-,%+

(a) FAILURE TO MEET OBJECTIVES

WHAT HAPPENED VERSUS SHOULD HAVE HAPPENED


Step 1


Pr%b(") R"&%*+,-,%+


WHAT HAPPENED VERSUS SHOULD HAVE HAPPENED (b) OPPORTUNITIES

WHAT HAPPENED VERSUS COULD HAPPEN


Step 1


Pr%b(") R"&%*+,-,%+





(2) Define the ProbLEM

Step 1


Pr%b(") R"&%*+,-,%+






VALIDATE SYMPTOMS WITH PRIMARY AND SECONDARY RESEARCH

Step 1


Pr%b(") R"&%*+,-,%+




SITUATIONAL ANALYSIS




Step 1


Pr%b(") R"&%*+,-,%+





(2) Define the ProbLEM SITUATIONAL ANALYSIS

VALIDATE PROBABLE CAUSES VALIDATE SYMPTOMS WITH PRIMARY AND SECONDARY

RESEARCH

Step 1


Pr%b(") R"&%*+,-,%+






VALIDATE PROBABLE CAUSES

DETERMINE INFORMATION GAPS


Step 1


Pr%b(") R"&%*+,-,%+








(1) STAKEHOLDERS (2) PROBLEM SYMPTOMS (3) POTENTIAL CAUSES (4) COURSES OF ACTION? !" '-.-")"+- '/%#(0

,+&(#0":


Step 1


Pr%b(") R"&%*+,-,%+






(3) CREATE RESEARCH OBJECTIVES





Problem Statement

Step 1


Pr%b(") R"&%*+,-,%+







HYPOTHESIS TO CLOSE THE INFORMATION GAP IN THE PROBLEM STATEMENT





Problem Statement

Step 1


Pr%b(") R"&%*+,-,%+







HYPOTHESIS TO CLOSE THE INFORMATION GAP IN THE PROBLEM STATEMENT

IE. To gather information from a sample representative of the PEEL REGION POPULATION AMONG THOSE WHO ARE “VERY LIKELY” to use an emergency shelter within the next 6 months, as

measured on a scale of 1 = Very UNLIKELY and 5 = VERY LIKELY. This data would serve as an input into a forecasting model to predict service demand.





Problem Statement

Step 1


Step 2


Step 3

Data Analysis and

Measurement

!" S#rv"$ Pr%&"''

Step 2


R"'".r&/ D"',*+

(a) EXPLORATORY BROADLY DEFINE THE PROBLEM

Step 2


R"'".r&/ D"',*+

(a) EXPLORATORY BROADLY DEFINE THE PROBLEM

EXPERIENCE SURVEYS CASE ANALYSIS

FOCUS GROUPS

SECOND-ARY

DATA

Step 2


R"'".r&/ D"',*+

(a) EXPLORATORY

(b) DESCRIPTIVE ANSWERS WHO, WHAT, WHERE, WHEN, WHY, & HOW

BROADLY DEFINE THE PROBLEM

Step 2


R"'".r&/ D"',*+

(a) EXPLORATORY

(b) DESCRIPTIVE ANSWERS WHO, WHAT, WHERE, WHEN, WHY, & HOW

CROSS-SECTIONAL STUDY

SAMPLE SURVEY

LONGITUDINAL STUDY PANELS

BROADLY DEFINE THE PROBLEM

Step 2


R"'".r&/ D"',*+

(b) DESCRIPTIVE

(c) CAUSAL IF X, THEN Y

(a) EXPLORATORY

ANSWERS WHO, WHAT, WHERE, WHEN, WHY, & HOW BROADLY DEFINE THE PROBLEM

Step 2


R"'".r&/ D"',*+

(b) DESCRIPTIVE

(c) CAUSAL IF X, THEN Y

(a) EXPLORATORY

BEFORE-AFTER WITH CONTROL GROUP

ONE-GROUP, BEFORE ONE GROUP, BEFORE-AFTER ANSWERS WHO, WHAT, WHERE, WHEN, WHY, & HOW BROADLY DEFINE THE PROBLEM

Step 2


R"'".r&/ D"',*+

(b) DESCRIPTIVE

(c) CAUSAL

(a) EXPLORATORY

ANSWERS WHO, WHAT, WHERE, WHEN, WHY, & HOW BROADLY DEFINE THE PROBLEM IF X, THEN Y

A''"'':

Step 2


R"'".r&/ D"',*+

A''"'':

(1) INTERNAL VALIDITY

(2) EXTERNAL VALIDITY

ANSWERS WHO, WHAT, WHERE, WHEN, WHY, & HOW BROADLY DEFINE THE PROBLEM IF X, THEN Y

(b) DESCRIPTIVE

(c) CAUSAL

(a) EXPLORATORY

Step 2


S#rv"$ S-r#&-#r"

Step 2


S#rv"$ S-r#&-#r" Question type Location Examples rationale

Step 2



Screens Part 1 Have you visited a

shelter in the past month? Selection

Step 2



Warm-Ups

Screens Part 1

Part 2

Have you visited a shelter in the past month?

To What extent were you satisfied with the experience?

Selection

Ease

Step 2



Transitions

Warm-Ups

Screens Part 1

Part 2

Part 3



The following part of this survey concerns your employment history.

Selection

Ease

Subject change

Step 2



Complicated Questions

Transitions

Warm-Ups

Screens Part 1

Part 2

Part 3

Part 4




Rate each of the following factors on a scale of 1 = no need to 7 =

extremely need.

Selection

Ease

Subject change

Commitment

Step 2



Complicated Questions

Classification and Demographics

Transitions

Warm-Ups

Screens Part 1

Part 2

Part 3

Part 4

Part 5




Rate each of the following factors on a scale of 1 = no need to 7 =

extremely need.

What is the highest level of education you’ve attained.

Selection

Ease

Subject change

Commitment

Personal

Step 2


Q#"'-,%+ F%r)#(.-,%+

Step 2


Q#"'-,%+ F%r)#(.-,%+

Step 2


Q#"'-,%+ F%r)#(.-,%+

Thin

gs to

do

Step 2


Q#"'-,%+ F%r)#(.-,%+

Thin

gs to

do

•  Be Focused •  please rate your experience

with x on the following features.

•  Be Brief •  Did you use the emergency

shelter as a last-resort?

•  Use Simple Structure •  To what extent do you rely

on emergency shelters? (Quantify options)

•  Be Crystal Clear •  Under What Circumstances

would you find the emergency shelter useful? (LIST OPTIONS)

Step 2


Q#"'-,%+ F%r)#(.-,%+

Thin

gs to

do









Thin

gs to

Avoi

d

Step 2


Q#"'-,%+ F%r)#(.-,%+

Thin

gs to

do









•  Leading q’s •  Shouldn’t everybody have

the right to live free of poverty?

•  Loaded Q’s •  Have you quit smoking?

•  Double Barreled Q’s •  Do you think social service

agencies should offer more educational resources and outreach programs?

•  Overstated Q’s •  Do you think This program

will eliminate poverty in Peel?

Thin

gs to

Avoi

d

Step 1


Step 2


Step 3

Data Analysis and

Measurement

!" S#rv"$ Pr%&"''

Step 3

DATA ANALYSIS AND MEASUREMENT

S&.(" C/.r.&-"r,'-,&'

(a) NOMINAL SCALES LABELS OBJECTS (RACE, RELIGION, TYPE OF DWELLING, AGE, GENDER,

YES/NO Q’s) DESCRIPTION

Step 3


S&.(" C/.r.&-"r,'-,&'



(b) ORDINAL SCALES INDICATE RELATIVE SIZE DIFFERENCES

BETWEEN OBJECTS (SERVICE-FREQUENCY USAGE)

DESCRIPTION + ORDER

Step 3


S&.(" C/.r.&-"r,'-,&'





DESCRIPTION + ORDER

(c) INTERVAL SCALES DESCRIPTORS ARE EQUAL DISTANCES

APART (RATING A SERVICE ON ITS OVERALL QUALITY FROM POOR = 1 and

EXCELLENT = 5) DESCRIPTION + ORDER + DISTANCE

Step 3


S&.(" C/.r.&-"r,'-,&'





DESCRIPTION + ORDER

(c) INTERVAL SCALES DESCRIPTORS ARE EQUAL DISTANCES

APART (RATING A SERVICE ON ITS OVERALL QUALITY FROM POOR = 1 and

EXCELLENT = 5) DESCRIPTION + ORDER + DISTANCE

(d) RATIO SCALES TRUE ZERO ORIGIN EXISTS (# OF

$ SPENT ON X PER WEEK, DISTANCE TRAVELLED, # OF CHILDREN, # OF PEOPLE IN

HOUSEHOLD) DESCRIPTION + ORDER +

DISTANCE + ORIGIN

Step 3


D.-. T1" .+0 S-.-,'-,&'

Step 3


D.-. T1" .+0 S-.-,'-,&'

DATA Relationships Statistical Analysis

Step 3


D.-. T1" .+0 S-.-,'-,&'


•  CROSS-TABULATIONS •  CHI-SQUARE TESTS (PRESENCE OF RELATIONSHIP) •  CRAMER’S V-STATISTIC (STRENGTH OF RELATIONSHIP)

(a) NOMINAL (CATEGORICAL) SCALES

Step 3


D.-. T1" .+0 S-.-,'-,&'



•  SPEARMAN RANK-ORDER CORRELATION •  KENDALL’S TAU


(b) ORDINAL (RANKING) SCALES

Step 3


D.-. T1" .+0 S-.-,'-,&'



•  SPEARMAN RANK-ORDER CORRELATION •  KENDALL’S TAU

•  SCATTER PLOT •  PEARSON PRODUCT MOMENT CORRELATION


(b) ORDINAL (RANKING) SCALES

(b) METRIC (INTERVAL/RATIO) SCALES

Step 1


Step 2


Step 3

Data Analysis and

Measurement

!" S#rv"$ Pr%&"''

QUESTIONS?

Warren Seuradge [email protected]

Marketing

Survey Instruments and Questionnaire Design