Research Skills One Oh, no! Please, no! Not statistics!!

Research Skills One

Oh, no! Please, no! Not statistics!!

Aims of the two first year Research Skills courses:

Basic grounding in research skills -

designing questionnaires and experiments

data analysis using statistics, Excel and SPSS

writing up studies using standard presentation conventions

Outline of this course (Research Skills One):

Issues in designing questionnaires and experiments.

Descriptive statistics - effective summarising of data.

Statistical tests (correlation, Chi-Square, nonparametric tests for comparing groups or conditions).

Using SPSS ("Statistical Package for the Social Sciences") for data analysis and Excel for producing graphs and tables.

Organisation for this course:

One lecture and one practical session every week.

(Lectures: Fridays at 10 am in Chichester Lecture Theatre until week 12 – December 5th).

Practical sessions - details on Sussex Direct (you are split into lots of smaller groups).

Course requirements:

Assessment is by a mixture of coursework and exam:

Coursework –SONA research participation (4 hours/16 credits this term): 10%

Lab-report 1 10%Lab-report 2 20%

Exam - Unseen exam (in winter assessment period): 60%

Overall pass mark (exam and coursework combined): 40%

Course documentation, handouts, lecture slides:

Go to Study Direct - this course is

“Research Skills One” (C8511)

OR

Type “Graham Hole” into Google

Lecture slides available on Study Direct the day before the lecture.Lecture podcasts available on Study Direct the day after the lecture.

Click on this link

Everything you need to know for this course

Questionnaire design:

Perhaps now you'll fill in my questionnaire?

How does the way you collect data affect the data collected?

How does question wording affect people's answers?

Stages in questionnaire design:

5. Administer the questionnaire, after it has been revised in the light of (4).

1. Formulate the research question(s) clearly.

2. Identify the population and sample.

3. Design the questionnaire: think about question wording; questionnaire formatting; mode of administration; data analysis.

4. Pre-test the questionnaire.

Samples and populations:Sample: a subset from a population (e.g. first-year psychology students).

Population: a complete set of things (e.g. all of humanity).

For valid inferences to be made about a population's characteristics, a sample must be representative of its parent population (e.g. similar in age, SES, IQ, etc.)

Methods of obtaining questionnaire data:

Postal questionnaire

Personal interview

Phone interview

Internet questionnaire

Cost Low High Moderate Low

Data quality:

Response rate

Respondent motivation

Interviewer bias

Low

Low

None

High

High

Moderate

Moderate

High

Low

Low

Low

None

Sample quality: Low, unless high response rate

High Moderate to high, especially with random digit dialling

Moderate, but improving as Internet access widens

Postal Questionnaire

Personal interview

Phone interview

Internet questionnaire

Possible interview length:

Short Very long Long Short

Ability to clarify and probe:

None High High None

Anonymity: High Low Low Low

Dependence on respondent’s literacy:

High None None High

Control of context and question order:

None High High Depends

Goals of Questionnaire design:

1. To obtain facts about a person.

2. To obtain information about their attitudes and beliefs.

3. To find out what a person has done (behaviours).

Questionnaire wording:

1. Should be exact.

2. Should be simple.

3. Avoid biased or emotive words.

Schuman and Presser (1981): subtle changes of wording may influence responses.

e.g. “Should the Government allow public speeches by a Communist?” produced 25% fewer pro-free-speech responses when allow was replaced with forbid.

4. Make all alternatives clear.

e.g. Payne (1951): "Do you think most manufacturing companies that lay off workers during slack periods could arrange things to avoid layoffs and give steady work throughout the year?"

63% - companies could avoid layoffs.22% - couldn’t avoid layoffs.15% - no opinion.

Same question plus phrase "…or do you think layoffs are unavoidable?"

35% - companies could avoid layoffs.41% - couldn’t avoid layoffs.24% - no opinion.

5. Avoid the format: "Some people say x: do you agree or disagree?"

6. Avoid unwarranted assumptions. e.g. "What is your occupation?" assumes person has a job.

7. Avoid double-barrelled questions. e.g. "Should immigrants be repatriated and their possessions confiscated?" is two questions.

8. Avoid double negatives. e.g. "Are you against a ban on smoking?"

9. Consider the relative merits of open-ended and

closed-ended questions.

Open-ended: allow unconstrained responses.

e.g. "How do you travel to the University?".

May produce richly detailed responses, but hard and tedious to score.

Closed-ended: require choice from a limited range of alternatives.

e.g. "Do you travel to the University by

(a) bus, (b) car, or (c) unicycle (tick one)".

Easy to code, but prone to bias.

Closed-ended questions must have

(a) a balanced response scale;

(b) mutually exclusive categories;

(c) facilities for handling "don't know" and "other" responses.

Rating scales:

The Likert Scale:

"Criminals should be flogged".

Strongly agree Agree Neither agree nor disagree

Disagree Strongly disagree

1 2 3 4 5

Can be 5-. 7- or 9-point scale (doesn't make much difference).

Visual Analogue Scale:

Strongly agree Strongly disagree

Problems with questionnaires about attitudes:

1. May not have an attitude - "doorstep opinions".

2. Attitudes may be complex and multi-dimensional.

3. Attitudes vary in intensity.

4. Expressed attitudes may depend on question wording, sequence and interviewer effects.

Problems with questionnaires about behaviour:

1. Memory limitations -

e.g. Henry et al's (1994) study of 18 year old's memories for mood at age 9-11; Chapman and Underwood's (2000) study of drivers' memory for accidents and near-misses.

Can be counteracted by

(a) asking specific questions;

(b) asking for birth date rather than age;

(c) using a chronological format;

(d) re-interviewing.

2. Response biases due to social desirability or suspicion, especially for illegal or anti-social activities.

Can be counteracted by ensuring anonymity.

5. Do your children prefer to eat in KFC or Macdonalds?

1. Do you visit fast food emporia regularly?

2. How many burgers do you eat per month?

3. Some people suggest that fast food is leading to increased tooth decay and an increase in obesity amongst teenagers in many parts of the U.K. Do you agree?

4. Which of the following methods do you use to travel to your fast-food outlet? (a) Bus (b) Car (c) Bicycle

What are the problems with these questions?

5. Do your children prefer to eat in KFC or Macdonalds?

1. Do you visit fast food emporia regularly?

Over-complex wording. Exactly what does "regularly" mean?

2. How many burgers do you eat per month?Assumes you eat burgers.

3. Some people suggest that fast food is leading to increased tooth decay and an increase in obesity amongst teenagers in many parts of the U.K. Do you agree?

Over-long. Includes two separate questions. Implies you should agree with the views expressed.

4. Which of the following methods do you use to travel to your fast-food outlet? (a) Bus (b) Car (c) Bicycle

Does the questioner want you to choose only one option, or can you choose more? No option for responding "other". Assumes you go to fast-food outlets.

What are the problems with these questions?

Assumes you have children.

Conclusions:

It's hard to design questionnaires properly!

Always be sceptical of survey results -ask yourself

Who were they collected by?

Who were they collected from?

How were the questions worded, exactly?

Remember - “8 out of 10 cats prefer Whiskas" has become "In tests, 8 out of 10 cats who expressed a preference, preferred Whiskas".

Useful references (on my website):

Burgess, T.F. (2001). A general introduction to the design of questionnaires for survey research.

Taylor-Powell, E. (1998). Questionnaire design: asking questions with a purpose.

Faculty Facts - answers to School Office by TODAY

Win a £25 voucher and bottle of champagne!!

(Lost the sheet? Download it from Study Direct).

Which of these people…

-worked for Walt Disney before becoming a lecturer?

-is an amateur trapeze artist?

-likes kayaking and printmaking?

-goes birdwatching with Bill Oddie?

-is a Liverpool supporter?

-fell off the stage while presenting at a Health & Safety conference?

-loves paragliding?