1

Questionnaire Design

Epidemiology

P McKeown

EPIET Introductory Course 2003VERYIER DU LAC, FRANCE

2

So far………IntroductionOutbreak or IncidentSalmonella in Northern IrelandSteps in investigationSurveillance

Questionnaire Design

Disease Detectives Looking for clues

Building a case

3

ObjectivesTo understand:

Why we use questionnairesThe objectives of a questionnaireThe advantages and disadvantages of questionnairesThe different types of questions that can be used and the circumstances in which they would be usedThe way in which questionnaires can be designed and how to avoid some common problems and pitfalls

4

What is a questionnaire?An instrument (form) to collect answers to questionsCollects factual data – gathers information or measuresA series of written questions / items in a fixed, rational orderEvery item is answered (unless instructed otherwise)Who completes it?

The respondent – public/professionalSomeone on behalf of the respondentThe interviewer

5

Types of Questionnaire

Prospective or retrospectiveCompleted by participant or researcherAdvantages of self-administered include:

Cheap and easy to administerPreserve confidentialityCan be completed at respondent's convenienceCan be administered in a standard manner

6

Types of Questionnaire

Advantages of interview-administered include:

participation by illiterate peopleclarification of ambiguity

They can be administered face to face, via the telephone, by post or email

7

What makes an effective questionnaire?

An effective questionnaire is highly structured

to allow the same types of information to be

simply collected from a large number of

people in the same way so that the data can be

analysed quantitatively and systematically

8

And why do we want one?

A well designed questionnaire:Will give you appropriate data which should allow you to answer your research questionWill minimise potential sources of bias, thus increasing the validity of the questionnairePeople are much more likely to complete a well designed questionnaire

9

AssumptionsThe use of questionnaires is based on

certain assumptions:That individuals will have had common understanding of particular issues and will share general beliefs

(went to a wedding/ that we are trying to find out more about meningitis because this disease is a threat to our children and young people)

That this understanding and these beliefs are shared and can be clearly expressed and quantified

10

Before starting…..Questionnaire design should be an integral part of the study, not an “add-on”Therefore:

Decide on the goals of the study (for yourself and for others)Know the subject – literature, expertsKnow the respondents - (SEG, occupation, special sensitivities, educational, ethnic)Know how much – cost, time, sample size, response rate

11

Fundamental Principle

An epidemiological investigation is a scientific loop that tests a hypothesis:

Problem Hypothesis

Study

Questionnaire

Question

Results

Answer

12

Objectives of a questionnaireTo maximise the proportion of subjects answering the questionnaire –as high a response rate as possible To ensure that the information obtained is as accurate and as relevant to our study as possible

= as simple and as focussed as possible

13

Advantages of questionnaires

Can reach a large number of people relatively easily and economically (especially postal questionnaires)Provide quantifiable answersRelatively easy to analyse

14

Disadvantages of questionnairesMay provide only limited insight into the problem:

Limited response allowed by questionsYou may not have asked the right questions eg Andorra, norovirus and ice-cubes

Varying response they may not understand your question and may intrepret it differently

Postal questionnaires low response rateHard to chase after missing data – need to get it right first time

15

Stages in designing a questionnaire

Decide on aim of the study and the purpose of using a questionnaireDecide on content of the questionsDecide on the format of the questionsDecide on presentation and layoutFormulate coding schedule (if appropriate)Pilot and refine questionnaire

16

Basic RulesKISS keep it short and simpleAppearance is crucial and affects:

Response rateEase of data summarisation and analysis

Length of questionnaire: shorter response ratesQuestion order is important:

Easy difficultGeneral particularFactual abstractStart with closed format questions. Start with questions relevant to the main subject. Try to avoid starting with demographic and personal questions

17

Basic Rules

Number all items and pagesPut an identifying mark on all pagesPut the return address on the questionnairePut the study title in bold on the first pagePrint directions in bold

Remember the Unique Identifier

18

Basic RulesGroup questions by topic or response optionsDon’t put the most important item lastOpen with relevant non-threatening questionsIt is most likely to be completed if easy on the eye, relevant, logical and as short as possible.

19

Content of Questions

Keep a clear focus on your research question Literature search; what do we need to know in order to ask the most relevant and probing questions?Demographic information Contact information (if non-anonymised)

20

Format of QuestionsAsk only for the information you needStart with simple questions – get them on your sideAppropriate for responding audience – eg middle class wedding guests vs prisonersKeep sentences simple and short

21

Format of QuestionsAsk for one piece of information at a time:

If you had not passed water for 6 hours but were watching a James Bond movie, and your wife brought you a sandwich would you get up to go to the toilet in the next 20 minutes? Y/N

Remember “don’t know”

22

Format of QuestionsBe precise – Do you often have to get up at night to pass water? Y/NBe appropriate/sensitive – bowel habitsAim always to minimise bias

23

Bias in QuestionsInformation Bias = systematic differences in the measurement of a response

Recall bias – eg cases being more likely to remember than controlsObserver bias – inter and intra – a questionnaire is an important way to reduce this

Non-response biasThose who respond are different from those who do notReduce by ensuring high response rate orCorrecting for during analysis (eg age, sex, SEG)

24

Format of questions Two main question formats

Closed format give a ‘fixed’ responseOpen format allow people to express their views in their own words:

What is your most distressing symptom? Please describe: ________________________________________________________________________________________________

25

Open or Closed?Closed – forced choice

Simple and quick Reduces discrimination against those who are less literate Easy to code, record, and analyse results quantitatively Easy to report results

Open format – free textAllows exploration of issuesCan be used even if a comprehensive range of alternative choices cannot be compiled

26

Closed QuestionsA number of types:

1. Straightforward response1. What is your age in years? _____ yrs

How long have you worked for Company X? _____ What is your sex (gender)? M F Did you stay in Hotel X on 23/6/02? Y N Did you eat dinner in the company canteen

on 23/6/02? Y N

27

2. Checklist

Which of the following did you eat for dinner on 23/6/02?• Chicken • Beef • Ham • Salad • Egg mayonnaise

Closed Questions

28

Closed Questions3. Adjectival/adverbial responses

Are you limited by hip pain during any of the following activities?

Not at all Slightly Moderately Severely

Walking Sitting

Lying in bed

29

Closed Questions4. Numerical rating scale

How useful would you think a community methadone clinic in your area would be? (please circle)

1 2 3 4 5 6 7Not at all useful Very useful

30

Closed Questions5. Visual analogue scale

At rest, how bad is your hip pain?(please mark where on the line below)

No pain Worst painimaginable

31

Closed Questions6. Ranking scales

Please rank these activities in order of the amount of pain they cause you. (1 would be most pain and 4 would be least)

Sitting Running Lying in bed Walking

32

Closed Questions7. Scales for measuring attitude eg Lickert

A local methadone clinic would be useful for my community

No, I strongly disagree No, I disagree quite a lot No, I disagree just a little I’m not sure about this Yes, I agree just a little Yes, I agree quite a lot Yes, I strongly agree

33

Open QuestionsUsed to provide fuller informationOpen, free-text formOften used in qualitative research, focus groups, at the preliminary and piloting stage of questionnaires

What do you think is the reason you became ill? ________________________________________________

Do you think service X would make your life easier? If so, in what way? If not, why not? ________________________________________________________________________________________________

34

Problems and PitfallsAvoid questions that ask two things at once - you won’t know which ‘bit’ people are answering:Have you ever had neck pain and diarrhoea?

Ambiguity..... e.g. Do you got to the toilet a lot?

35

Problems and PitfallsAvoid jargon

How often do you get up at night to PU? (pass urine)Should IVDUs be treated in the community?

Avoid options that are not mutually exclusive when they need to beWhat age are you?

16-20 20-25 25-30 35-40

36

Problems and PitfallsAvoid leading questionsDo you think that the food in the hotel made you sick?Did the hotel staff seem unhygenic to you? Do you agree that the hospital staff were close to

exhaustion?

Avoid making the questionnaire too longTypographical / spelling errors

QuestionaireDiahurriaVommitEppidemiology

37

Ensuring a High Response RateMake the questionnaire relevant

Ensure respondents see its potential and usefulness (we are trying to find out why you and your family/people in became ill while holidaying in this resort)

38

Ensuring a High Response RateInclude a brief covering letter

Who you areWho you work forWhy you are investigating/researchingWhere you obtained the respondent’s nameHow and where you can be contactedAbsolute guarantee of confidentialitySelf-addressed envelope!!

39

Questionnaire ValidationMany existing questionnaires can be used or adapted eg SF-36ValidatedOthers e.g. for an outbreak have not been validated and will need to be testedPiloting is a common way of doing this

40

Piloting and EvaluationPilot with a similar group of people to your intended subjectsPiloting a essential

Highlights problems before startingOften concentrates initially on effects of alternative wordingThen will concentrate on respondents and interviewers overall impressionsWith a final polishing after several amendments to ensure response rate

41

Presentation and layoutDecide on the order of the items / questionsWhere to place sensitive questions?Make the layout clear, consistent and easy on the eye

Try to get on overall simple “feel”Ensure there is adequate space to answerChoose a large enough font sizeAvoid flippancyAvoid fancy logos/printed on recycled paper/is an equal opportunity employer etcDo not slip to another page

42

Presentation and layout Using colour or printing questionnaire on coloured paper may help

Use filter questions, if necessary

Give clear instructions about how to answer the questions

43

Coding ScheduleQuestionnaire can be pre-coded Quicker and easier to hands responses prior to data entry eh Epi-Info or SPSSExamples:

M=1 F=2 DK=3Ill=1 Not ill=2 DK=3Single=1 Married=2 Separated=3 Divorced=4 DK=5No medical intervention=1 Saw GP=2 Went OPD=3 Admitted Hospital=4 Admitted ICU=5 Died=6 DK=7

44

SummaryA well designed questionnaire:

Will give you appropriate data which should allow you to answer your research questionWill minimise potential sources of bias, thus increasing the validity of the questionnairePeople are much more likely to complete a well designed questionnaire

45

FINALLY, keep your questionnaire short and the questions simple, focussed and appropriate

Question 764 …“If you are an attractive, blonde woman aged 25-50, can I take you to dinner?”