Survey Research at WCU

Hal Herzog

The truth about the subject pool:• Little incentive to give you good data.

• Usually young and inexperienced.

• Often have a haphazard approach to surveys.

• Compromise the validity and reliability of your results.– Source of unwanted variation in data

(experimental error)– Reduce the effect size of your results

The truth about experimenters:• Often unconsciously give the message

that the research is not important.– Show up late.– Don’t show up at all.– Treat subjects unprofessionally.– Undergraduate assistants are not trained.– Sloppy signage on rooms.

• Surveys are often poorly designed.

My experience with surveys:

• Galvin and Herzog (1998) – animal rights demonstrators– Handed out 750 surveys– Got 231 back (33% return rate)

• Scott Plous (1997) – psychologists– Sent surveys to 5,000 APA members– Got 3,982 back (80.4% return rate)

• Plous and Herzog (2001) – Animal care committees– Sent 556 surveys to university faculty members– Got 494 back (87% return rate)

Tailored Design MethodDon Dillman

• Mail and Internet Surveys: The Tailored Design Method

• Based on social exchange theory (Thibaut and Kelly 1950)

• It works – increases typical return rates from 25% to 75%.

• TDM principles apply to getting better data from WCU Subject Pool

• Basic information in Chapters 1 – 3.

Some Principles of Treatment of Subjects:Reward and Reciprocity

• Show positive regard – introduce yourself– ask their name

• Say “Thank You” – in advance.• Give tangible rewards – in advance.

– Certificate of participation?• Make the questions interesting.• Sponsorship by legitimate authority.• Make task appear important.• Ask for advice.

Some Principles of Survey Design:

• Booklet format• Cover – have a picture on it.• Introductory letter.• Order of questions.• Pay attention to details – e.g. font size,

bold, italics, etcetc.• Provide a navigational path.• Use back page for open-ended question.

Some Principles for Writing Questions:

• Use complete sentences.• Choose simple over specialized words.• Use as few words as possible to pose the

question.• Use mutually exclusive response categories.• Separate “undecided” from “neutral” on Likert

scale items.• Included an open-ended question at the end.

Other tips.

• Consent form should be user-friendly.– Question/answer format?

• Pilot studies are critical.– Assess question clarity.

• These details make a difference!