SurveyMonkey Audience Survey Writing Guide

How To Be A SurveyMonkey Audience Pro at Writing SurveysImpress your friends, colleagues and clients with your survey skills—look no further for all the tips and best practices to create a great SurveyMonkey Audience survey

Table of Contents

Survey Writing Guide

Ask yourself WHY --------------------------------------3Stay relevant --------------------------------------------3Stay focused --------------------------------------------3Write the right question -------------------------------4

Speak their language ------------------------------------------4Tell them when --------------------------------------------------4Stay balanced ---------------------------------------------------5Keep it personal ------------------------------------------------5Be direct ---------------------------------------------------------6

Write the right answers --------------------------------6Use closed-ended questions ----------------------------------6Use your words -------------------------------------------------7Avoid yes / no questions --------------------------------------8Measuring pricing ----------------------------------------------8Measuring “how much” ----------------------------------------9Provide a full set of answers ----------------------------------10Watch your ranges ---------------------------------------------11Matrix / Grid questions ----------------------------------------11

Specific Settings Guidelines

Require all questions -------------------------------------------12Randomize answer choices when possible or relevant ---12Skip logic --------------------------------------------------------12Titles --------------------------------------------------------------12Progress bars ---------------------------------------------------12

Audience Pro Preferences-----------------------------12For Multiple Choice (Multiple answer) questions ----------12When similarly worded questions are asked, highlighting the

keywords that make them different helps keep respondents

focused. There are a few easy ways to do this: ------------12Use sentence case for all answer choices -------------------12

Survey Formatting Guide -----------------------------13Formatting Key -------------------------------------------------13References -------------------------------------------------------13

3Questions or want to learn more? Visit us at audience.surveymonkey.com or call 800-310-6838 |

Survey Writing GuideAsk yourself WHY

The first question you should ask yourself is why you are running a

survey to begin with. Asking yourself why you’re running a survey

will help you come up with one clear goal for your survey, like a

decision you need to make or an insight you’re trying to get. This

goal will guide you through the whole survey writing process and

will give you a place to refer back to if you get stuck.

Stay relevant

Once you have that big picture “why” of your survey, you’ll need

to break this main goal up into subtopics. Focus on key data

needs, not “nice to have” information. This keeps the survey

creation process simpler and makes data analysis easier. Make

sure your goal is very clear and your subtopics are measurable.

€ Tip from the Audience Pro: This will streamline the number of questions you need to ask, and speed up your analysis since you can focus on the things that matter most.

Example:

Goal: What does Netflix consumer usage look like?

Subtopic 1: How many people subscribe to Netflix?

Subtopic 2: How satisfied are subscribers with Netflix?

Subtopic 3: What are people using instead of Netflix?

Stay focused

Write out the questions to answer your goal and subtopics directly.

Working from your broad goal down to the evidence you need to

support it will make sure that the specific questions you ask relate

directly back to the reason you’re asking in the first place.

Example:

Question for Subtopic 1: Are you currently subscribed to Netflix?

Questions for Subtopic 2: Overall, how much do you enjoy the content available on Netflix? How likely are you renew your Netflix subscription at the end of your billing cycle?

Question for Subtopic 3: What other service(s) do you use to watch streaming video?

€ Tip from the Audience Pro: Don’t know how to ask your questions? Use the SurveyMonkey Question Library to find commonly used survey questions that are simply worded and easy to analyze.

audience.surveymonkey.com

http://help.surveymonkey.com/articles/en_US/kb/SurveyMonkey-Audience-Additions-to-the-Question-Bank-Library


Survey Writing GuideWrite the right question

Speak their language

Be careful when throwing around terminology that may be familiar

to only you or your industry, or complex ideas that other people

may not understand. Remember, you need to make sure the

people who are taking your survey understand what you’re talking

about. If they don’t understand your question, they might answer

it incorrectly, randomly, or not at all! Try to write your survey in as

plain language as possible. If you have to leave in a complex or

obscure word, providing a definition or some examples of what

you’re talking about can be helpful.

d Avoid: Do you own a tablet PC?

2 Use: Do you own a tablet PC (e.g. iPad, Android tablet)?

Tell them when

Limiting the scope of a question to a specific time period ensures

that all of your survey takers are thinking about the same scope

of time when they’re answering your question. If you don’t do this,

then you have no way of knowing whether someone is thinking

about the past day, the past week, or the past year. Moreover,

specifying a specific time period will make recall of past behaviors

and projections of future behaviors more accurate.

For example:

• Yesterday, did you…

• In the past week, have you…

• How many times in the past 30 days have you…

• In the next 12 months, will you…

If you’re not sure what time period you’re most interested in, or

you want an overall picture, you can begin questions with words

such as “In general,” “Overall,” or “Typically”, to ensure that your

respondents are thinking about their average behaviors. A word

of caution, however, that this is difficult for respondents to think

about and the answers you get back may not be a true average of

their feelings.

d Avoid: How much do you usually spend on groceries?

2 Use: In the past 7 days, about how much have you spent on groceries?




Stay balanced

When writing a question, try not to bias the respondent’s answer

by leading them in a certain direction. If possible, try to incorporate

both sides of an opinion within the question. (For example, “Do

you like spinach, or not?”) This will make sure that respondents are

comfortable expressing their true opinions. When it’s not possible

to balance within a question, try to balance the survey overall. Ask

questions that are framed in both a positive and negative manner

so that the tone of the survey is balanced as a whole. If you don’t

balance within questions or within the survey, respondents are

likely to think that you have a particular bias and this will impact

how they answer the questions. (Some may try to tell you what

they think you want to hear and others may be reactive, trying to

tell you the opposite of what they think you want to hear.)

d Avoid: How much have you increased your usage of Netflix in the past 3 months?

2 Use: Has your usage of Netflix increased, decreased, or stayed the same in the past 3 months?

Keep it personal

Have you ever heard the expression, “There’s no such thing as a

stupid question?” Well, you need to reassure survey respondents

that there’s no such thing as a stupid answer. Sometimes

respondents can feel a little intimidated by the idea of giving their

opinions on issues they don’t feel qualified to talk about. Make the

questions feel more approachable by keeping them personal. Use

cues like “do you think,” “do you feel,” “in your opinion” to signal to

respondents that you’re not looking for an absolute judgment here

and that there’s no “right answer”—all you want is their perspective

on things.

d Avoid: Is the price for a yearly Netflix subscription too high, too low, or about right?

2 Use: Do you feel that the price for a yearly Netflix subscription is too high, too low, or about right?




Be direct

The simplest question is usually the best. Don’t beat around the

bush—get straight to the point! Asking direct questions that are

written clearly and concisely will ensure that survey respondents

understand what you mean. This will help lower your dropout rate

and make sure that the data you get back is good data.

d Avoid: What was the state of the cleanliness of the room?

2 Use: How clean was the room?

3 Write the right answers

Use closed-ended questions

When writing a survey question, think about your main goal of the

question. If you want to know the main reason behind respondents’

actions, or their topmost opinion on something, allow only one

answer. Open text questions are helpful to get a thorough list of

keywords that people think of about certain topics. However, using

single-response, closed-ended questions will make your questions

easier to answer and your analysis process quicker and easier.

d Avoid: Why did you decide to cancel your Netflix subscription?

2 Use: What was the main reason you canceled your Netflix subscription?

• Price was too high

• Didn’t like the content

• Don’t have time to watch Netflix content anymore

• Other (please specify)

NOTE: We do offer open-ended questions and “select all that

apply” options for all of our questions. However, use these

question types with caution, as the data you get can be difficult

to analyze and interpret. Therefore, as a general rule of thumb, we

recommend against allowing multiple answers.



Survey Writing Guide3 Write the right answers

Use your words

People tend to think in words, not numbers—so you should ask

them to rate their opinions on scales that have words not numbers.

Rating scales (also known as Likert scales) should have 5 points if

they are unipolar or 7 points if they are bipolar. You should use a

unipolar scale as your default, and a bipolar scale ONLY if a clear

midpoint exists and the adjective at the top is a clear and direct

opposite of the adjective at the bottom.

For example…

Unipolar: the response can be thought of as having a five-point scale (1 to 5).

• Extremely

• Quite

• Moderately

• Slightly

• Not at all

Bipolar: the response can be thought of as having a seven point scale (-3 to +3). Use this with ratings that are exact opposites of each other (e.g., short and tall, hot and cold).

• Much more

• Moderately more

• Slightly more

• Neither more nor less

• Slightly less

• Moderately less

• Much less

d Avoid: Please rate 1 to 5 how much you enjoy the content available on Netflix.

• 1 - Enjoy a lot

• 2

• 3

• 4

• 5 - Enjoy a little

2 Use: How enjoyable is the content available on Netflix?

• Extremely enjoyable

• Quite enjoyable

• Moderately enjoyable

• Slightly enjoyable

• Not at all enjoyable




Avoid yes / no questions

Yes/no questions demand that your survey takers think in black

and white—but in the real world, most opinions are shades of gray.

Sure, a yes/no question can be valuable if you need a straight-

forward answer to filter responses by, or if you want to split your

respondents into different groups in order to disqualify certain

respondents. However, in most cases, offering survey takers a

range of options will make them feel more comfortable selecting

an answer choice that accurately reflects their opinion. It will also

make your analysis easier and more powerful to have a variety of

answers for respondents to choose from.

d Avoid: Will you be upgrading to the iPhone 5 in the next month?

• Yes

• No

2 Use: How likely are you to upgrade to the iPhone 5 in the next month?

• Extremely likely

• Quite likely

• Moderately likely

• Slightly likely

• Not at all likely

Measuring pricing

Some of the most frequently asked questions for our Audience

customers are questions about the pricing of their product or

service. For these questions, follow the same rules we’ve talked

about above and focus on giving people a range of responses, and

think about whether you want more objective information (such as:

“How much would you be willing to pay for a video subscription?”)

or more subjective information such as: “Do you think our prices

are too high, too low, or about right?”). Take questions that only

give a respondent two options and turn them into a range that

allows people to give you a more nuanced answer.

d Avoid: What do you think of the current price for a Netflix subscription?

• I like it

• I don’t like it

2 Use: Do you think $100 for a yearly Netflix subscription is high, low, or about right?

• Extremely high

• Quite high

• Somewhat high

• About right

• Somewhat low

• Quite low

• Extremely low




Measuring “how much”

Want to know how much respondents spend their time, money,

or energy on something? Just ask. There are many different ways

to ask how frequent a certain behavior is. You can measure their

subjective perceptions (“How often do you go to the gym?”) or

their objective actions (“How many times a week do you go to

the gym?”). One may be more useful than the other depending on

your research goal so think about which one you choose. If you

choose to measure subjective perceptions, use a rating scale like

we mentioned above; if you choose to measure objective actions,

make sure you limit your response options to no more than 6. Also,

providing ranges for each answer option will help people with

fuzzy memories find a response that feels right for them.

Example (subjective):

About how much of your marketing budget do you spend on online advertising?

• All of my marketing budget

• Most of my marketing budget

• About half of my marketing budget

• Some of my marketing budget

• None of my marketing budget

Example (objective):

About how much of your marketing budget do you spend on online advertising?

• 0-20%

• 21-40%

• 41-60%

• 61-80%

• 81-100%




Provide a full set of answers

When providing answer options, make sure your list is exhaustive

and has all possible answer choices that a respondent could

choose. But, in the rare case that you think you might not have

allanswer choices down or cannot remember all applicable options,

providing an alternative for respondents who cannot find a suitable

answer choice is a good idea. An answer choice that can be helpful

in accomplishing inclusivity is:

Other (please specify)

d Avoid: In the past 30 days, what site have you used most to watch streaming TV shows?

• Netflix

• Amazon Prime

• Hulu Plus

2 Use: In the past 30 days, what site have you used most to watch streaming TV shows?

• Netflix

• Amazon Prime

• Hulu Plus


A lot of our customers add a “none of the above” option. We

recommend against including options such as “none of the above,”

“not applicable,” and “don’t know” as they provide an easy way out

for respondents not to answer the question regardless of whether

they have an answer. Instead, we recommend using skip logic! Skip

logic keeps respondents honest and engaged in your questions,

and ensures that your data quality stays high.

d Avoid: In the past 30 days, what site have you used most to watch streaming TV shows?

• Netflix

• Amazon Prime

• Hulu Plus

• None of the above

2 Use 2 questions:

In the past 30 days, have you watched streaming TV shows?

• Yes [if yes, display next question]

• No [if no, skip next question]

In the past 30 days, what site have you used most to watch streaming TV shows?

• Netflix

• Amazon Prime

• Hulu Plus





Watch your rangesWhen using ranges in a multiple-choice (single answer) question,

there are three main things to remember. First, make sure that

they don’t overlap. Overlapping ranges will confuse respondents,

decreasing data quality. Next, keep the range intervals the same

size. This will make your data analysis go more smoothly as you

can use the midpoint of the range to look at trends in your data. In

general, keeping intervals constant makes any statistical analyses

you run that much stronger. Finally, make sure your list of response

options spans the whole possible range of responses. Again, as we

talked about before, 5-7 response choices are ideal depending on

what is helpful for you.

d Avoid: In a typical day, how much time do you spend watching content on Netflix?

• 1-2 hours

• 2-5 hours

• 5-10 hours

2 Use: In a typical day, how much time do you spend watching content on Netflix?

• Less than 2 hours

• 2-4 hours

• 5-7 hours

• 8-10 hours

• More than 10 hours

Matrix / Grid questions

Organizing similar questions into a matrix is visually appealing and

easy. However, displaying questions in this manner actually hurts

your data quality in two ways. First, having questions in a matrix

allows respondents to get lazy and sloppy, clicking quickly down a

column of responses without actually engaging with the questions

you’re asking. The repetitive nature of a matrix question in a

survey gets respondents bored fast. This also increases dropout

rate, which is bad news for you. Additionally, not only do matrix

questions hurt your data quality because your respondents aren’t

engaged, it also hurts your data quality because you, as the survey

creator, are not as engaged. Matrix questions allow you not to

think critically enough about how you’re asking the questions. Most

questions shouldn’t have identical response options.

What do you do instead? We recommend breaking down your

matrix question into a separate question for each row with

customized responses. We charge you by the row for a matrix

question anyway, so it won’t cost you a penny more, and it will

keep your data quality and your response rate high!



Specific Settings GuidelinesRequire all questions

Survey respondents might skip a question by mistake, or click the

“next page” button too early, so requiring answers to questions can

help avoid missing key data.

Randomize answer choices when possible or relevant

Remove order bias by randomizing answer choices in multiple

choice questions. Flip the order of scales. Whenever possible

randomize questions on a page and even pages in the survey.

Skip logic

Skip logic is a great way to customize your respondents’ survey

experience and make sure that they only see questions that are

relevant to them. Make it easy on yourself—label all pages involved

in skip logic paths to be sure you’re skipping your respondents to

the right places.

Titles

If including survey titles and page titles would bias responses they

should be hidden, however, if possible, try to include them. Titles

help to organize the narrative and flow of your survey, and provide

a quick sense of order and orientation for your respondents.

Progress bars

We recommend against including progress bars as they are

distracting for respondents. You want to keep respondents

focused on the questions you’re asking, not how fast (or slow)

they’re going.

Audience Pro Preferences

The following tips are SurveyMonkey Audience Survey Expert

preferences for consistency purposes. They are internal guidelines

and not necessarily needed to ensure methodological soundness,

but can lead to better respondent experience.

For Multiple Choice (Multiple answer) questions

At the end of the question, always add: (Please select all that apply.)

When similarly worded questions are asked, highlighting the keywords that make them different helps keep respondents focused. There are a few easy ways to do this: 3 Underlining sticks out

3 So does bold 3 ALL CAPS work too

Use sentence case for all answer choices 3 Capitalize only first letter of each option

3 This does not apply in the case of proper nouns



Specific Settings GuidelinesSurvey Formatting Guide

€ Tip from the Audience Pro: If you are programming your survey offline, use this formatting guide to make it easy for you or anyone else to understand and upload your survey into SurveyMonkey. This is not necessarily to ensure Methodological soundness, but will be helpful when you program your survey into your SurveyMoneky account.

Formatting Key 3 Items in [BRACKETS] are instructions for inputting the survey into SurveyMonkey, examples include:

• [randomize] = randomize responses

• [skip to PAGE TITLE] = skip that answer choice to PAGE TITLE

• [skip to DISQUALIFY] = skip that answer choice to DISQUALIFY

o if [skip to DISQUALIFY] is used on a PAGE TITLE, use page logic

to skip all to DISQUALIFY

• [open text] = use an open text or paragraph question type

3 Items on their own line, in ALL CAPS, indicate that a new page should be added, and the page title of the new page

• SPLIT PAGE indicates to add a new page or use the “split page” button

3 Numbered items (e.g., 1. Lorem ipsum...) indicate questions

3 Specific formatting guidelines

• When a question ends with “(Select all that apply.)”, this question

should be a multiple choice, multiple answer question type

• When needed, the “add Other as an answer choice” option should be

checked while inputting the survey into SurveyMonkey

• When [randomize] is used, if the last answer choice, or second to last

answer choice when an “Other (please specify)” option is used, should

not be randomized (this is an option in the relevant question types)

• If words in a question or answer choice use underline or bold

formatting, use HTML in the question for the designated words or

phrases

o for question language, use underline, since all questions are

bolded: <u>word or phrase</u>

o for answer language, use bold: <b>word or phrase</b>

3 All questions should be required unless otherwise noted

3 Hide asterisks for required questions

3 Hide all page titles

3 Hide all question numbering

References 3 How to add Other as an answer choice

3 How to require questions

3 How to hide page titles and question numbers

3 How to enable HTML in your account


http://help.surveymonkey.com/articles/en_US/kb/How-do-I-add-an-other-textbox-or-comment-field

http://help.surveymonkey.com/articles/en_US/kb/How-can-I-make-survey-questions-mandatory

http://help.surveymonkey.com/articles/en_US/kb/Can-I-hide-the-survey-title-or-page-titles-from-respondents

http://help.surveymonkey.com/articles/en_US/kb/Can-I-use-HTML-in-the-survey-or-Email-Invitation-message

Self Improvement

SurveyMonkey Audience Survey Writing Guide