Research SkillsWeek 3: Questionnaire Design

We went through the research process◦ Original Article vs. Secondary source◦ How to find and read a paper◦ Scientific writing style◦ Research Treasure Hunt◦ Maths questionnaire

This week: Issues with questionnaire design

Last Week

Sampling Consent Question types Answer types Questionnaire design Coding Issues with questionnaires handout Fast food questionnaire

Today

Who to ask: your target population How many people Avoid a biased sample, e.g. if asking about

drinking behaviour in men and women:◦ Don’t just ask women◦ Don’t just ask people in a bar◦ Don’t just ask tee-totallers

Sampling

You must adhere to a strict code of ethics in your research:

http://www.bps.org.uk/sites/default/files/documents/code_of_ethics_and_conduct.pdf Participants must:

◦ give consent to take part◦ not be coerced into participating◦ be free to withdraw at any time

Consent

Participants create their own answers◦ “What is your age?”

◦ “Are you a smoker?”

◦ “What are your favourite TV programmes?”

◦ “How much do you like biscuits?”

Question Types: Open-Ended

Experimenter provides participants with options◦ Choice of category:-

Are you a smoker? Never smoked / Current smoker / Ex-smoker

◦ Likert scale: - How strongly do you agree with the statement “I like biscuits”

1 2 3 4 5Strongly Disagree Disagree Neutral Agree Strongly

Agree◦ Checklists: Circle the TV programmes that you watch◦ Rating scales: How much do you like this drink, on a scale of 1-

10?

Question Types: Closed-Format

Open-Ended◦ Exploratory◦ Useful when you can’t cover all the possible

answers◦ Impractical in terms of analysis

Closed-Format◦ Easy and quick to fill in ◦ Doesn’t matter how literate or articulate you are◦ Easy to code, record, and analyse results

quantitatively ◦ Easy to report results

Advantages and Disadvantages

If the answer to the question is a number that represents an amount, e.g.◦ IQ score◦ Height◦ How long it takes to complete a jigsaw puzzle◦ Likert scale responses

Top tip: Calculating a mean makes sense with continuous data (but not with categorical data)

Answer Types: Continuous Data

Please give an approximation of the number of alcoholic drinks you normally consume on a Saturday night: … Drinks

Please indicate your agreement with the following statement:◦ I feel that I should drink less on a Saturday night □ □ □ □ □

1 2 3 4 5Strongly Disagree Neutral Agree StronglyDisagree Agree

Examples of Continuous Data

If the answer to the question is:◦ a word

“Yes”◦ a description

“Physics student”◦ a code that represents a category

1 = undergraduate, 2 = postgraduate

NB: Numerical codes can be used to represent categorical responses BUT this does not transform categorical data into continuous data

Answer Types: Categorical Data

In which town were you born? …….

Please indicate your gender:□ Male □ Female

Which actor is the hunkiest?□ Brad Pitt□ Johnny Depp□ Orlando Bloom

Examples Categorical Data

“Please indicate your age:”◦ Continuous: … Years◦ Categorical : □ 18-25 □ 26-30 □ 31–35 □ 36–40 etc.◦ Categorical : ... Years □ Older than 60 Years

“How many days a week do you usually exercise?”◦ Continuous : … days◦ Continuous :

□ 1 day □ 2 days □ 3 days □ 4 days □ 5 days □ 6 days □ 7 days

◦ Categorical : □ 1 day □ 2 days □ 3 days □ 4 days □ 5 days or more

This can be applied to a number of data

Examples of Tricky Bits of Data

Be short and simple Start with an introduction/ welcome

message Allow not applicable responses to all

possibly relevant questions Say thank you to participants

A questionnaire should…

Go from general to particular Go from easy to difficult Go from factual to abstract Not start with demographic and personal

questions (put these at the end)

A questionnaire should…

Assure anonymity◦ Assign each questionnaire a number instead of

asking for names Avoid personal and sensitive questions Be aware that you may bias answers simply

by being there Avoid biased wording

◦ e.g. “Would you agree that the death penalty is a bad idea?”

A questionnaire should…

Giving numbers to categories in categorical data is called coding◦ e.g. “Yes” becomes 1 and “No” becomes 2

Codes can be allocated either before the question is answered (pre-coding) or afterwards (post-coding)

Coding

Work through the ‘Issues with questionnaire design’ handout ◦ Decide whether the questions are categorical or

continuous ◦ Code categorical questions ◦ Decide whether or not the questions are problematic ◦ Suggest how you would rectify any problems

Top tip: Not all the questions are problematic!

The Handout

Open “Fast food study” on Graham’s website

Read the introduction and survey

Fast Food Questionnaire

2 Continuous Hypotheses: - ◦ Should be tested with a categorical question and

the continuous question (Purchases).

◦ e.g. “Males consume a larger quantity of fast food per month than females”

Fast Food Questionnaire

In groups:

Discuss what categorical questions might affect fast food purchases (based on last week’s research)

Come up with two hypotheses

Next week: entering and analysing data in SPSS

Fast Food Questionnaire