Questionnaire Design

1

Facilitated by Elizabeth Wiredu

2

Course Aim

The aim of this course is to facilitate the design of valid questionnaire to support good data collection and the subsequent analysis.

3

Learning OutcomesBy the end of the course, you will be able to:

Identify the principles that will guide the design of a good questionnaire.

Learn how to put a good design together.

Examine how response types determine the choice of summaries and statistics you could generate on your data.

Critique a poorly designed questionnaire.

Create a sample good questionnaire from scratch.

4

Questionnaire

What is questionnaire?

When to use questionnaire?

Is validated questionnaire the solution?

The Framework

Pose the research question. Develop and write the questionnaire,

and do validity checks. Summarise and analyse data. Interpret and report findings.

6

Stage 1- Pose the Research Question

Funnel the research question down into specific key

questions or objectives which will then be grouped as

themes.

Write down the hypotheses and the conclusions you

want to arrive at.

Establish the relationships, comparisons, trends,

patterns or summaries you wish to make.

Stage 2- Develop and Write the Questions

Formulate each key question into a theme and choose which variables you must collect data on.

Decide how each variable is to be measured i.e. how to frame the question (open/closed) and its response set.

Get help from your literature reviews.

Stage 2- Questions Designs

Factual Information Simple dichotomy Simple numeric Simple choice

Opinion and Attitude Information Likert categorical choice Anchor scale Rank order

Mixed Designs Multiple choice Matrix design

9

Examples of Closed Questions Design

Dichotomous

Questions???Response:

Yes No Male Female

Simple Choice

Questions???Response:

Exceeded expectations Met expectations Fell short of expectations

Likert Scale

Questions???

Response: Very satisfied Somewhat satisfied Undecided Somewhat dissatisfied Very dissatisfied

Anchor Scale

Questions???1-directional design: On a scale of 1 to 10, where 1 is virtually no pain and 10 is excruciating pain.

2-directional design:On a scale of 1 to 5, where 1 is highly unsatisfied and 5, highly satisfied.

10

Examples of Closed Questions Design

Multiple Choice

Which of the following products do you own? (select all that applies)

Pen drive Computer Printer Fax machine Scanner Digital camera

Matrices

Please rate your purchase decision based on the following characteristics

V. Important Will Consider Not Important

Reliability Price Features The look Functionality

11

Opened Question Design Example

Open ended question…???

Do you think smoking and wound-healing are linked, and if so, how? Please write your response in the box below.

Stage 2- Questions Designs

WordingPhrase your questions in positive toneLogical Arrangement, Appearance, and LayoutAmbiguity and ImprecisionAcronyms, Jargons, Technical LanguageAssumptionMemory and KnowledgeDouble Questions or complex and heavy loaded questionsLeading QuestionsWording Answers before the QuestionsSubjective QuestionsVague Questions

13

Reliability and Validity

Reliability. A questionnaire is considered to be reliable if the results of a study can be reproduced at different times using the same questionnaire under a similar methodology.

Use test-retest method to check reliability

Validity refers to whether the questions (questionnaire) are measuring what they are supposed to measure.

Validity checking: use pre-test

14

Stage 2- Sampling

How much sample do you need?.

Decide on the appropriate Sampling Method(s)

Simple Random

Stratified Random

Systematic Sampling

Cluster Sampling

Quota Sampling.

15

Pay attention to the following during the planning stage

Have you budgeted for non-response by sending (10-40%) more to protect your specified sample size for your power calculation?

Have you carefully considered the ethics of the research, e.g. questions you should not ask? The protection of the respondents’ data?

Have you cleared access to your respondents?

Are you going to use software to summarise and analyse the responses? If so, do you know how to use the software or have you got access to support?

16

Pay attention to the following design issues:

Themes and their logical arrangement Place a text box under each theme to ask for an overall opinion about the theme Avoid ‘Any comment/suggestion’ question at the very end. It does not work! Clear instructions to reduce response error Navigation – adopt question numbering and goto jumps, etc. Appearance, layout and sequence Logical, inclusive/complete response sets

17

Piloting the Questionnaire

Proofreading and EditingPre-test pilotingFeedback prompts

Do you believe other people may have difficulty in answering this

questionnaire? If so, which question in particular may pose a problem?

Given the purpose of my study outlined in the introduction section, would you

suggest a question, may perhaps be added or removed to enhance the study?

How long did it take you to complete the questionnaire?

Did you read the instructions?

Were the instructions clear and neatly emphasised?

Did you object to answering any of the questions?

Was the layout of the questionnaire clear, logical and attractive?

Is there any question that the wording could be enhanced or clarify?

18

Summarise and organise the data

Look for common and unusual patterns, trends,

differences, features in the data

Relate the data to the research question

Undertake appropriate statistical analyses of the data

Clearly communicate the relevant findings

Access the limitations of the study

Stage 3- Summarising and Analysing the data

Stage 4- Interpretation and Reporting of Findings

Relate the findings to the research questionDraw appropriate conclusionsCompare and contrast findings with those of other relevant researchSuggest future research question leading on from here.

Introduction to the Questionnaire

A short introduction explaining:

The purpose of your research.

How this study will benefit the respondent directly or indirectly.

State official backing or sponsorship if any.

A note of appreciation to the respondent for their contribution to your study.

Confidentiality if applicable and Data protection statement.

Contacting information if respondent wish to make a contact.

The introduction letter MUST be part of the questionnaire and also the separate covering letter.

Concluding Note

21

A short concluding note containing:

A note of appreciation to the respondent for their contribution to your study

Returns address

Returns deadline

22

Frequently Asked Questions Question: Can I use tick boxes for all the questions? Answer: It depends on your intended statistical analysis. E.g. if you

intend to do a comparative testing (Ttest/ANOVA) you will need the dependent variable to be a continuous data (scale or ordinal measure) and the factor variable, categorical data (choice response).

Question: After the data is collected, can it be manipulated to allow certain statistics?

Answer: It is possible to regroup continuous variables into categorical, or even reduce categories of a categorical variable to take advantage of certain statistics. However, it is not possible to convert categorical variable to continuous variable.

23

Evaluate a sample questionnaire

Group Feedback and Discussion of the sample questionnaire

Create a questionnaire from scratch

Examine the design of your questions and the summaries or statistical analysis opportunities available.

What Next? - Practical Session

24

How can I help you?

We provide:

Questionnaire design feedback.

A guide to summary statistics and inferential statistical analysis.

Assistance with reporting the statistics for publication.