Upload
suzanne-akemi-sannwald
View
114
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
CREATING AN EFFECTIVE SURVEY INSTRUMENT
SUZANNE SANNWALD
ASHFORD UNIVERSITY
INSTRUCTIONAL DESIGN & DELIVERY
EDU 652
DR. KATHY ZIENTEK
MAY 4, 2013
CREATING AN
EFFECTIVE ONLINE SURVEY
INSTRUMENT
1. Online Survey Benefits2. Survey Process3. Survey Goals: What, Who, Why & How4. Online Survey Design Hints5. Designing Questions
1. Writing Effective Questions
2. Survey Question Types
3. Question Sequencing
6. Feedback7. Results Processing
Overview
Online Survey Benefits
Participation can be self-select, if openly available on a Web page
Participation can be targeted when distributed by e-mail
Quick turnaround, although potentially lower response rates
Cost efficient, particularly with large samples
Optional anonymity conducive to questions of a sensitive nature
(Schoniau & Fricker, 2002, p. xiv, viii-xx)
What are the basic steps of survey creation?
Click icon to add pictureWhen creating online surveys, “the basic steps in the process
should be the same as with any other survey
medium” (Schoniau & Fricker, 2002, p. 6).
Survey Process
Plan optimal survey timing considering time necessary for entire process
University of Wisconsin-Madison (2010) defines the survey process as follows:
An additional step is to apply findingsApply findings
(University of Wisconsin-Madison, 2010, p. 4)
Survey Goals: What
What is the purpose of the survey? What do you want/need to learn?” According to Barlow (2010), surveys often
have a goal of gathering information about: “Feelings, attitudes, reactions to products or
services” “Anticipated reactions to new kinds of products
or services” “Current levels of behaviors/activities for
planning of or to gauge impact of programs” (p. 3).
Survey Goals: Who
Who is your target population? Who will you use as your sample
(subgroup of your target population)? Sample Size Calculator
Reasons to use a sample: If the target population is too large or
widely dispersed To allow for more reminders and follow-up
contact Consider any relevant privacy issues
(Barlow, 2010, p. 4)(University of Wisconsin-Madison, 2010, p. 5, 17)
Survey Goals: Why & How Why do you need the information about
the particular target population? How will the results help you or others? How will you use the results? How can you apply your findings?
Will you use information to change current or future service delivery?
Will you re-evaluate the impact of any changes?
(Johnson, n.d., p. 3)
How else can we improve response rates?
Click icon to add pictureFrom the start, improve response rate by giving respondents
“clear indication of intent of survey and how information will
be used”(Barlow, 2010, p. 9).
Online Survey Design Hints
Ensure and communicate protection of privacy, as appropriate
Provide indication of survey progress Limit questions per screen Limit use of graphics, understanding graphics may cause
bias Limit use of complex or matrix questions Restrict and validate responses when possible to reduce
response errors Add and test all skip logic to streamline survey Only require responses when necessary Ease navigation with format (e.g., spacing, fonts, shading) Thoroughly test survey, including technical compatibility(Schoniau & Fricker, 2002, p. 41-53)
How can we improve reliability and validity?
Click icon to add pictureStrive for reliability and validity:• “Reliability is the extent to
which repeatedly measuring the same property produces the same result” and
• “Validity is the extent to which the survey question measures the property it is supposed to measure”
(University of Wisconsin-Madison, 2010, p. 6).
Writing Effective Questions
Survey goals should drive and shape questions According to Johnson (n.d.), surveys reflect a
solely personal expression and should thus avoid: “Asking secondhand information” “Asking hypothetical questions” “Asking about causation” “Asking for solutions to complex issues” (p. 14)
Language used should be familiar to respondents Do not use abbreviations Avoid technical language and jargon
Writing Effective Questions
“Questions are
effective only if all learners understand them and can answer
them the way you
intended”
(Horton, 2012, p. 243)
Focus on one issue per question Use simple, direct, neutral language
Keep both questions and choices short Avoid difficult, ambiguous, or leading language Avoid questions in the negative or with double
negatives Include clear instructions
Specify conditions such as time frames (e.g., within the past week)
Specify tasks required (e.g., select one, select all that apply)
Emphasize important word (e.g., “not”) Use parallelism with answer choices
Same length Equivalent grammar
(Barlow, 2010, p. 5, 7)(Horton, 2012, p. 244-
259)(Johnson, n.d., p. 11-
13)
What types of questions are there?
Click icon to add pictureTo pick the right question
type, “consider what kind of information
you are trying to gather” (Horton, 2012, p. 220).
Survey Question Types
May take more time for respondents to answer
Takes more time to process
May provide new or unique insight
Responses may be hard to interpret and analyze
Quick response time
Respondents are more limited in responses
Quick processing time
Insights are limited response options
Open-Ended “Fill in the blank” Responses
Closed or Structured Responses
(University of Wisconsin-Madison, 2010, p. 9)
Survey Question Types: Closed
Examples: Multiple Choice True/False Checklists Attitude scales
Options should be exclusive (non-overlapping)
Options should be exhaustive Include an “Other” option when appropriate
For frequency, use specific number ranges versus “often, seldom, never”(Barlow, 2010, p. 5-6)
Question Sequencing
Order questions for a logical flow Start with an easy question Start with questions that reflect the study topic
and/or pique interest Group similar questions For questions requiring recall, use the order in
which they happened Sequence questions about details before asking
for an overall assessment End with personal and/or demographic questions Enable forward and backward navigation through
questions (Johnson, n.d., p. 18)(University of Wisconsin-Madison, 2010, p.
11)
Feedback
During Survey: Provide indication of survey progress Provide clear error/warning messages Provide an avenue for respondents to report
problems. Incomplete Surveys:
Follow-up with reminders to complete surveys Completed Surveys:
Send thank you messages for survey completion Consider providing incentives for survey completion Update respondents with survey results Plan outreach to respondents based on survey
results (Schoniau & Fricker, 2002, p. 45, 47, 52-53)
Results Processing: By Survey Consider audience and purpose of result
sharing Define demographic characteristics of
sample Report survey response rate Determine handling of incomplete
surveys Provide general summary of results Attach survey as appendix
(Barlow, 2010, p. 13)
Results Processing: By Question Quantitative Processing:
For open-ended responses Report key themes and their frequency
Qualitative Processing For closed responses Report frequency, percentages, and by
categories Find ways to visually represent
responses
(Barlow, 2010, p. 12)
References
Barlow, P. (2010). Effective survey design and analysis [Presentation]. Retrieved from University of Wisconsin-La Crosse: http://www.uwlax.edu/catl/Presentations/T4TW/Effective%20Survey%20Design%20and%20Analysis+Worksheet%20-%20June%202010.pdf
Horton, W. (2012). E-Learning by design (2nd ed.). San Francisco, CA: Wiley.
Johnson, K. (n.d.). An introduction to effective survey design and administration [Presentation]. Retrieved from Penn State University Survey Research Center: http://studentaffairs.psu.edu/assessment/pdf/KurtJohnsonPresentationFA11.pdf
University of Wisconsin-Madison Office of Quality Improvement. (2010). Survey fundamentals: A guide to designing and implementing surveys. Retrieved from http://oqi.wisc.edu/resourcelibrary/uploads/resources/Survey%20Guide%20v%202.0.pdf
Schoniau, M., Fricker, R. D., & Elliott, M. N. (2002). Conducting research surveys via e-mail and the Web. [Adobe Digital Editions version]. Retrieved from http://site.ebrary.com/