Gathering Content Insights from User Surveys

Gathering Content Insights from User

SurveysConfab Central 2014

t: @Kathy_CS_Inc #ContentSurvey #ConfabMN

Kathy Wagner: Mom (and grandma!) Tai chi master (yeah, right) Not quite a crazy cat lady (yet…)

Kathy Wagner: Technical Communicator User Experience Researcher Content Strategist

“We cannot solve our problems with the same kind of thinking we

used to create them.”~ Albert Einstein


Why use surveys for content projects?

Conduct research quickly, easily, and affordablyGives your audience a voiceAligns stakeholders and streamlines decision-making




It’s your responsibility too

Content Strategy

Customer Experience


Audience Attitudes + Content InsightsGoals TopicsPriorities FormatsLikes & dislikes ChannelsComparisons Terminology

t: @ Kathy_CS_Inc #ContentSurvey #ConfabMN

Surveys are GOOD for gathering insights to make better content decisions based on user need and interest.

1. Focus your research questions2. Choose your sample population3. Choose your incentive4. Design the survey5. Distribute the survey6. Analyse & communicate the results


6 Steps to Effective User Surveys

“Focusing is about saying ‘No’.”

~ Steve Jobs


Don’t begin your survey until you know:1. Exactly what you want to find out.2. From who.3. How you’re going to use the information.

Focus your research questions


Don’t begin your survey until you know:1. Exactly what you

want to find out.2. From who.3. How you’re going to

use the information.


Do you want to find out:What terminology to use?What topics to cover?What formats to publish?Or…?






Do you want information about:All your audiences?Specific audiences?Sub-set of an audience?






How are you going to use the information?Don’t gather any information that you don’t have a clear and immediate use for!



What kind of content do our web users want?

Example:

Which news article topics and formats do our alumni audience want to see on our homepage?


“‘Stalking’ is such a strong word. I prefer ‘intense research of an individual.’”

~ Sandrine, in Girl From Paris


Choose your survey sample


Choose your survey sample

Tips for choosing your survey sample:Random sampling is better.Don’t survey people more than 4 times per year.Target 400 responses, or as many as possible.


“Nobody made a greater mistake than he who did

nothing because he could only do a little.”

~ Edmund Burket: @ Kathy_CS_Inc #ContentSurvey #ConfabMN

Choose your incentive

Ideas for budget-conscious incentives:$5 or $10 Starbucks gift cards.Enter names into a draw for an iPad or ?Free or discounted product or service.


“If I had an hour to solve a problem and my life depended on it, I would use the first 55 minutes determining the proper questions

to ask.”~ Albert Einstein

t: @ Kathy_CS_Inc #ContentSurvey #ConfabMN

Design your survey: Structure

Tips for structuring surveys:Include opening and exit text. Divide the survey questions into categories.Organize the questions strategically.




Opening text should include:Name of your organizationConfidentiality of dataEstimated time to completeInformation about incentiveWhy their feedback is important




Exit text:Thank people for their timeReiterate importance of feedbackRemind them about incentive




Example categories:1. Audience tasks and

goals2. Topics of interest3. Formats of interest





Broad & Easy

Specific & Sensitive

1. Screening questions

2. Easy questions3. Difficult questions4. Sensitive

questions

Design your survey: Question types

Question types to consider:Open-ended questionsMultiple choiceRanking orderRating scale




Open-ended questions allow participants to answer freely.

Example:What are two ways that we could improve our videos?




Multiple choice questions force participants to choose one or more answers from a predefined list.

Example:Are you a US Citizen? (Y/N)t: @Kathy_CS_Inc #ContentSurvey #ConfabMN



Ranking order questions ask participants to rank answers in priority against each other.




Example:Please rank the following kinds of articles, from least important to most important:

Personal profilesBrief news articlesDetailed research articles

1st 2nd 3rd




Rating scale questions ask participants to rate the importance of various options.To consider:Numbered scalesWord scalesUsing neutral scores




Example numbered scale:On a scale of 1 – 7, with 1 being awful and 7 being amazing, how would you rate your overall experience with our knowledge base articles?




Example word scale:How satisfied are you with our knowledge base articles?o Very satisfiedo Satisfiedo Neither satisfied nor

dissatisfiedo Dissatisfiedo Very dissatisfied




Understanding neutral scores:#1) I would rather watch a real-life video than an animated video.o Strongly agree (5)o Agree (4)o Neither agree nor disagree (3)o Disagree (2)o Strongly disagree (1)




Understanding neutral scores:#2) I would rather watch a real-life video than an animated video.o Strongly agree (4)o Agree (3)o Disagree (2)o Strongly disagree (1)o No opinion (void)


Design your survey: Question typesThis question type Is best for

Open-ended Attitudes and opinions or when options are unknown.

Multiple choice Demographic info, or all feedback with a limited set of options.

Ranking Comparing importance between up to five options.

Rating number scale Rate the importance of items within a limited set of options.

Global or ESL audiences when survey can’t be translated.

Rating word scale Most rating questions and audiences.

Rating w/o neutral score

Gathering insights to make decision between two options.


Design your survey: Phrasing questions

Do not:

Lead respondents to an answer Use emotional languageUse double-barrelled questions Use extreme absolutesProvide vague or subjective options

Do:

Be briefBe grammatically simpleUse simple, familiar terms



Do not:

Lead respondents to an answer BadStaying involved in your professional society is important. How often would you like to receive news updates?

BetterHow often would you like to receive news updates?


Hey, You:What’s a better

question?

Do:


Do not:

Lead respondents to an answer

Do:

BadStaying involved in your professional society is important. How often would you like to receive news updates?

BetterHow often would you like to receive news updates?



Do not:

Use emotional language

BadHow likely are you to share life-saving safety information with your loved ones?

BetterHow likely are you to share safety information with your family members?


Do:

What’s a better question?


Do not:

Use emotional language

BadHow likely are you to share life-saving safety information with your loved ones?

BetterHow likely are you to share safety information with your family members?


Do:


Do not:

Use double-barreled questionsBadWould you like to be notified about new articles through Facebook and Twitter?

BetterWould you like to be notified about new articles? (Y/N)Choose how you’d like to receive notifications:


Do:



Do not:

Use double-barreled questionsBadWould you like to be notified about new articles through Facebook and Twitter?

BetterWould you like to be notified about new articles? (Y/N)Choose how you’d like to receive notifications:


Do:


Do not:

Use extreme absolutes

BadDo you always read articles from beginning to end?

BetterDo you read articles from beginning to end?


Do:



Do not:

Use extreme absolutes

BadDo you always read articles from beginning to end?

BetterDo you read articles from beginning to end?


Do:


Do not:

Provide vague or subjective optionsBad rating scaleo Frequentlyo Sometimeso Rarely

Better rating scaleo Dailyo A few times per weeko Once a week or less


What’s a better rating scale?

Do:


Do not:

Provide vague or subjective optionsBad rating scaleo Frequentlyo Sometimeso Rarely

Better rating scaleo Dailyo A few times per weeko Once a week or less


Do:


Question 5 4 3 2 1Satisfaction

How satisfied are you with… Very satisfied Satisfied Neither satisfied

nor dissatisfied Dissatisfied Very dissatisfied

Agreement

State your level of agreement with…

Strongly agree Agree Neither agree

nor disagree Disagree Strongly disagree

Extent To what extent do you… To a large extent

To a moderate extent

To some extent To little extent Not at all

Helpfulness How helpful is… Very helpful Somewhat

helpful Neither Not so helpful

Not at all helpful

Interest Indicate your degree of interest in…

Considerable interest

Moderate interest Some interest Little

interest No interest

Importance

How important to you is…

Very important

Somewhat important

Neither important nor unimportant

Somewhat unimportant

Very unimportant

Quality Please rate the quality of… Excellent Above

average Average Below average Poor

Tips for phrasing questions using a 5-point scale


Tips for minimizing survey bias:Provide list options in random order. Keep questions and answer options short and simple. Allow respondents to answer anonymously. Allow respondents to skip questions.

Design your survey: Minimize bias


Tips for maximizing response rate:Pre-notify survey sample of an upcoming survey. Pilot test your survey for clarity and functionality. Keep the survey short. Aim for 5 – 10 minutes. Offer incentives. Schedule reminder messages.

Design your survey: Maximize response


“You’ve got to kiss a lot of toads before you find your prince.”

~ Proverb


Distribute the survey through:Email lists (Monday mornings work best)Your websiteCompany newslettersSocial network channelsRelevant industry, professional, or social networks or clubsPrinted announcements that staff can hand to customers

Distribute your survey


“They’ve done studies, you know.

60% of the time, it works every time.”

~ Brian Fantana, in Anchorman


Tips for analysing and communicating survey results:Remove invalid responsesChoose the best chart typeFilter to compare responses between audience segmentsProvide context

Analyse and communicate the results



This chart type Is best for

Vertical bar chart Comparing percentages of 2 – 7 different groups.

Horizontal bar chart

Comparing percentages of 8 or more different groups

Pie chart Illustrating a sample breakdown.



Example: Vertical bar chart



Example: Horizontal bar chart



Example: Pie chart



Provide context. Example:

People want to find specific informationAll target audiences identified finding information quickly as the most important priority for them on the website. Having engaging and entertaining editorial content is a “nice to have”.


1. Focus your research questions2. Choose your sample population3. Choose your incentive4. Design the survey5. Distribute the survey6. Analyse & communicate the results


6 Steps to Effective User Surveys

Questions?Get more content tips at ContentStrategyInc.com

Find me on Twitter: @Kathy_CS_IncLinkedin: ca.linkedin.com/in/kathywagnercsSlideshare: Content_Strategy_Inc

Documents

Gathering Content Insights from User Surveys