8/14/2019 Week 4(2).pptx

1/41

8/14/2019 Week 4(2).pptx

2/41

Questionnaire

An important and widely used Research Method thatcan be very useful when done correctly

A quantitative approach aimed at finding information

that can be expressed in numbers, charts and graphs A mandatory approach for this course

8/14/2019 Week 4(2).pptx

3/41

Definition

The term Questionnaire refers to:

All methods of data collection in which different people

are asked to respond to the same set of questions in apredetermined order

8/14/2019 Week 4(2).pptx

4/41

Strengths

provide the possibility of surveying large andgeographically dispersed groups

are relatively low costs (answers may be submitted bypost)

Provide data that can be calculated and converted intocharts and graphs

Offer a lot of data in a limited time

Technology makes analysis quick and easy

There is reasonable anonymity for respondents

It can be completed at the time and location of therespondent's choice

It provides 'hard' figures which can be used to supportvarious decisions.

8/14/2019 Week 4(2).pptx

5/41

Weaknesses

Broad information, but little depth

Pre-structured response options and closedquestions can lead to a lack of nuanceanswersmust be restricted to options that do not allowdetail or grey areas

There is no room to adjust the process part way

through or to improvise. Even the smallest inaccuracy in the question

wording will make the data for the questionworthless

Some respondents will lie or provide sociallydesirable answers

8/14/2019 Week 4(2).pptx

6/41

Response rates

Different methods have different success rates

Online 10% (Internet) to 30% (within organisations)

Postal Around 30% is considered reasonable

Delivery and collection (actually picking it up) 30-50%

Face to Face / Structured Interview 50-70%

Each delivery method has advantages, but remember:

The success of your questionnaire is dependent upon thewill of respondents to participate.

8/14/2019 Week 4(2).pptx

7/41

What for?

Questionnaires are usually used for specific types of

information about the target group: Who they are (Personal attributes: age, gender, etc.)

What they think (Attitudes, Beliefs, Opinions)

What they know about a specific topic

What they do (behaviors and routines)

The right mix of questions can yield valuable info

8/14/2019 Week 4(2).pptx

8/41

Introduction

start with an introduction or a cover letterexplaining:

Who you are

what youre doing and why

what youll do with the information

8/14/2019 Week 4(2).pptx

9/41

Questions

Questionnaires have to result in information that canbe expressed in numbers. Coding

You will need to attach a numerical value (a number)to each answer choice

Yes = 1

No= 0, etc.

This is really important when it comes time to processyour data in a spreadsheet

8/14/2019 Week 4(2).pptx

10/41

Question Types:Open Questions

Open questions cannot be analysed with math. Keep them to aminimum or avoid them. If you want to ask mostly open

questions, do an interview or a focus group

Please, tell me how you feel about your new job

List 3 things you like about your job1 ..

2 ..

3 ..

8/14/2019 Week 4(2).pptx

11/41

Question Types:

Closed Questions

What do you like most about your job?

(Tick one alternative):

the salary (1)

thepeople you work with (2)

the opportunities for further schooling (3)

other, namely: __________________ (4)

8/14/2019 Week 4(2).pptx

12/41

Types of closed questions

Quantity questions

Dichotomy questions

Category questions

List questions

Rating scale questions

Ranking questions

Grid questions

8/14/2019 Week 4(2).pptx

13/41

Quantity

Sometimes you may want to ask respondents towrite down a number (their age, for example)

Use clear boxes to show where to fill in theanswer

You can use the numbers as codes

How many Children do

you have?

In what year were youborn?

5

1 9 8 5

8/14/2019 Week 4(2).pptx

14/41

Dichotomy

only 2 possible answers, which are usually opposites

Used for demographic data

Gender: male/female

Marital status: married/single

Sometimes used for attitudes and experiences Yes/No

Agree/Disagree

They are limited in scope

Please tick the box indicating your gender

Male (1)

Female (2)

8/14/2019 Week 4(2).pptx

15/41

CategoryCategory questions are questions that ask respondent toselect a single answer (called a category)

Used to find out about preferences, attitudes and behaviour

Only one category can be chosen

Theres a limited number of categories

The options must be mutually exclusive (no overlapping) The options must cover all possibilities

Which car colour would you prefer?

Blue (1)

Red (2)

Yellow (3)

Green (4)

Other, namely: __________ (5)

8/14/2019 Week 4(2).pptx

16/41

Lists (1)List questions are used whenever multiple answers are possible Often used to find out about a respondents experience, behaviour

or routine

You provide a list of possible answers

Note the coding in the example - this is actually five questions inone. You are asking people 5 yes/no questions and must be codedas such. Have you cleaned rooms, have you done the shopping,etc.

Please tickthe box in the providedcolumn for services you provided as ahome care volunteer in the past month. Multiple answers are possible.

Name of service Provided?

cleaning rooms (1,2)shopping (1,2)bed making (1,2)laundry (1,2)other (1,2)

(please describe:)

8/14/2019 Week 4(2).pptx

17/41

Lists (grid style)

Because lists involve essentially asking several questions at once,many researchers use a grid style for simplicity

Please tickthe box in the providedcolumn for services you provided as a

home care volunteer in the past month. Multiple answers are possible.

Name of service Provided?Yes (1) No (2)

cleaning rooms shopping

bed making laundry other

(please describe:) .

8/14/2019 Week 4(2).pptx

18/41

Rating ScaleSometimes you want to measure attitudes/opinions on a scale toallow more variety than yes/no, agree/disagree

These are used to find out about peoples knowledge and opinions

Note that its a good idea to include neutral opinions. Often the choiceis labeled, No opinion/I dont know or Neither agree nor disagree

For the following statement, please tick

the box whichmatches your view most closely.

Tourism should always be sustainable.

Strongly disagree (1)

Disagree (2)

No opinion/I dont know (3)

Agree (4)

Strongly Agree (5)

8/14/2019 Week 4(2).pptx

19/41

For the following statement, please tick thebox which matches your view most closely:

Tourism should always be sustainable.

agree tend to No opinion tend to disagree

agree disagree

(1) (2) (3) (4) (5)

5-point scale

8/14/2019 Week 4(2).pptx

20/41

Rating Scale (10 point scale for

detailed evaluation)

For the following statement please circle Othe

number which matches your view most closely

This trip was .

Poor value 1 2 3 4 5 6 7 8 9 10 Good valuefor money for money

10-point scale

8/14/2019 Week 4(2).pptx

21/41

Level of detail

In Rating scales you can decide how much detailyou want

Rough measurement

I agree(1)

No opinion (2)

I disagree (3)

More detailed

measurementI strongly agree (1)

I agree (2)

No opinion/I dontknow (3)

I disagree (4)

I strongly disagree (5)

Do you agree or disagree with the following statement:

CEOs of non-profit organisations shouldnt earn more

than the prime minister.

8/14/2019 Week 4(2).pptx

22/41

Neutral opinions?

When asking about attitudes and opinions, itsusually a good idea to have neutral options

Otherwise you risk manipulating respondents This could devalue results

Strongly agree (1)

Agree (2)

Uncertain (3)

Disagree

(4)

Strongly disagree (5)

Strongly agree (1)

Agree (2)

Neither agree nor disagree(3)

Disagree

(4)

Strongly disagree (5)

Do you agree or disagree with the following statement:

Penalties for driving offences should be much more severe.

8/14/2019 Week 4(2).pptx

23/41

Rank

Ranking questions allow you to ask people about their priorities

Used to find out what people think is important by having them rank

options. What do you look for in a new car? What political issues concern

you most? etc.

You can use the ranking numbers as codes

Please, number each of the issueslisted below in order of

importance to you.

Number the most important 1, the next 2 and so on.If a factor has no importance at all, please leave blank.

Factor Importance

Health 5

Education 2Human Rights 4

Child Care 1

Poverty 3

8/14/2019 Week 4(2).pptx

24/41

Frequency

Frequency questions refer to how often peopledo things. They are used to find out about

behaviour and routine.

The next questions concern your media habits.Please tick the box which most closely matchesyour experience.

I read one or more magazinesweekly (1)monthly (2)

every 3 months (3)every 9 months (4)less often (5)never (6)

8/14/2019 Week 4(2).pptx

25/41

Frequency (grid)

You can also use a grid to get information about frequency. Itenables you to ask multiple questions in a quick format that iseasy to fill in

For each of the following statements please tick the box which

most closely matches your experience

monthly every 3 every 9 less never

months months often

(1) (2) (3) (4) (5)

I read newspapers ..

I read one or more magazines

8/14/2019 Week 4(2).pptx

26/41

Things to keep in mind:

When asking closed questions, its important that alloptions be available.

But what if its not possible to include alloptions?Sometimes, its good to include the option:

Other, namely"

and allow respondents to fill something in

Keep in mind that you can code other, but will have torecord and analyse open answers separately

8/14/2019 Week 4(2).pptx

27/41

Other, namely (example)

Which car colour would you prefer?

Blue (1)

Red (2)

Yellow (3)

Green (4)

Other, namely: . (5)

8/14/2019 Week 4(2).pptx

28/41

Filtering

If your questionnaire has follow-up questions, youneed to provide clear instructions

If one answer determines what question therespondent should answer next, or if questions needto be skipped, make sure the respondent knows whatto do

Are you currently a member of an NGO?

Yes (1)

No

(2) If no go to question 24

8/14/2019 Week 4(2).pptx

29/41

Checklist for QuestionsEach question

Should only contain one question/variable. People oftenaccidentally ask multiple questions at once

Must be specific enough

Must be clear and unambiguous

Must not be too complex Must not be (doubly) negative

If you ask should smoking be allowed you get more trustworthyresults than if you ask should smoking be banned becausepeople naturally avoid negative phrasing

Starting questions with dont you think implies a desired answer

Double negatives (dont you think this is not a good idea?confuse respondents

Must not have an obviously correct or socially desirable answer(biased or leading questions)

Must not be too sensitive or unpleasant

8/14/2019 Week 4(2).pptx

30/41

Checklist for Answers

Each answer choice:

Must have the right level of detailnot too much or too

little Must be mutually exclusiveyou want to use answer

choices that rule each other out

Must be stated in a logical order

Must cover all possibilities

Must keep positive and negative options balanced

If necessary: include an option that is partly open, such as:

other, namely:

8/14/2019 Week 4(2).pptx

31/41

Good and Bad Examples

Bad examples

* It is not true that smokingdoesnt affect your health?

*Leading/negative

*Do you believe he lied whenhe denied his relationship withher?

*Biased/negative

*Do you think the teacher is anexpert in this topic?

* Is the respondent qualified?

*What is your opinion on thecontents and the assessmentof the course?

* 2 questions at once

Improved Examples

* Do you believe smoking isdangerous?

* Do you believe he was honest abouthis relationship?

* Do you think the teacher effectively

answers questions from the class?

* Make 2 questions:

* What is your opinion on the contentsof the course?

*What is your opinion on theassessment of the course?

8/14/2019 Week 4(2).pptx

32/41

Good and Bad examples

Bad example

*What percentage of your

weekly income do you spendon cigarettes?

*Requires calculation and will

take time

* It should be easy for

respondents to answerquestions

Improved example

*Make 2 simple questions

*What is your income permonth?

*Approximately how much

money do you spend on

cigarettes each week?

8/14/2019 Week 4(2).pptx

33/41

Good and Bad Examples

Bad examples

*Do you agree with the SurgeonGeneral's conclusion in his latest

report on cigarette smoking?* What if they didnt read the

report?

*Most doctors believe thatexercise is good for you. Do youagree?

* Implies a desiredanswer/judgmental

*When did you last see a movie?

* Vague

*How fast was car X going whenit smashed into car Y?

* Potential Bias

Improved examples

*Do you agree with the Surgeon

General's latest report thatcigarette smoking in publicplaces is never acceptable?

*Do you believe that exercise is

good for you?

*When did you last go to thecinema?

*How fast was car X going

when it hit car Y?

8/14/2019 Week 4(2).pptx

34/41

Good and Bad Examples

Bad examples

How often do you use

Blackboard?

Never (1)

Sometimes (2)

Always (3)

*Variables arent realistic

What is your monthly income?

Requires thought and could

make someone uncomfortable

Improved examples

How often do you use

Blackboard?Never (1)

Up to once a week (2)

Twice a week (3)

More

(4)

My income is:

Less than 30.000 (1)

From 30.000 up to 50.000 (2)

More than 50.000 (3)

8/14/2019 Week 4(2).pptx

35/41

Good and Bad Examples

Bad examples

How many times per month do

you play tennis?

1 to 5 times (1)

5 to 10 times (2)

more than 10 times (3)

These answers overlap

Which sports do you practise?

Soccer (1,2)

Tennis (1,2)

Field hockey(1,2)

Not all options are present

Improved examples

How many times per month doyou play tennis?

1 to 5 times (1)

6 to 10 times (2)

more than 10 times (3)

Which sports do you practise?

Soccer (1,2)

Tennis (1,2)

Field hockey (1,2)

Other: (1,2)

8/14/2019 Week 4(2).pptx

36/41

Questionnaire Lay-out

Introduction or cover letter

1. Explain who you are, what youre researching, and what youll dowith the answers

2. Is it anonymous?

Clear structure1. Put the questions into sections (by subject, for example) and

make sure theyre in a logical order

2. Use mini-introductions to provide context

It should be attractive and readable (fonts, etc.)

It should include a mix of question types It should be inviting it should encourage participation

It should be short

1. The longer it is, the less motivated the respondent

2. There should be no more than 20 questions, give or take

8/14/2019 Week 4(2).pptx

37/41

Cover Letter

Banner / logo Indicates topic and identifies

researchers

Date, place In full

Salutation Dear

Introduce the questionnaire State what the research is about,why it is useful

Indicate time Say how much time is needed to fill

it in (be aware that you want it to

seem easy)

What will you do with information? Stress confidentiality and

anonymity

Whatsit all for? State how results will be used

8/14/2019 Week 4(2).pptx

38/41

Pilot Testing

You write the questionnaire and get a small number ofpeople to fill it in for you

This will allow you to find any problems in the questionnaire(mistakes, badly worded questions, etc.) and to refine your

questionnaire You can observe or interview the people who fill the pilot

test in or you can add a few questions to the questionnairein which you ask for feedback about the questionnaire itself

Expert testing

Helps you determine representativeness of your sampleand suitability of your questions

Can help you see if the questions will lead to valid content

Testing with respondents who are in your target group Enables you to make sure the questionnaire suits that group

8/14/2019 Week 4(2).pptx

39/41

Pilot Test

In the pilot test you ask the testers about:

The clarity of the instructions

Which, if any, questions were ambiguous

Which, if any, questions the respondent felt uneasy

about answering

Whether in their opinion there were any major topic

omissions

Whether the layout was clear and attractive

Any other comments they might have

8/14/2019 Week 4(2).pptx

40/41

Pilot Test Report

A Pilot test report is a brief (1-2 pages)evaluation ofyour Questionnaire based on the results of your test

1. You evaluate your questionnaire

2. You state what you learned from the test

3. You explain any changes that need to be made /have been made as a result

8/14/2019 Week 4(2).pptx

41/41

What now?

Next week you will discuss your Research Plans andget feedback

But you need to get started on your questionnaire already

In Week 6 you will bring a draft of yourquestionnaire for feedback

During the break, you will conduct the pilot test

In Week 7 you will hand in the pilot test report