Embed Size (px)
Citation preview
14 Best Practices to Increase Survey Response Rates
Driver’s Manual 127
Decreasing response rates from surveys and a lack of participation in research efforts are presenting unique data challenges for companies across all industries. As the popularity and importance of data-driven decisions and the positive bottom-line impacts from Voice of Customer (VoC) efforts have grown, so has the number of companies prioritizing primary research. As a result, customers are often caught in a whirlwind of survey invitations from the variety of businesses they engage with on a daily basis (from feedback on a $2 coffee purchase to a multi-million dollar investment). Open rates, cooperation rates, and response rates have continually declined over the past decade because of this. This trend has spanned across both telephone and online research methodologies.
14 Best Practices to IncreaseSurvey Response Rates
Why are High Response Rates Important?
Gathering additional feedback and data offers stronger reliability and better flexibility. Stronger reliability relative to ensuring your proportionate sample of survey responses truly matches the population you are surveying as a whole. Is there a magic number your company should aspire to? The short answer is no because of the number of variables which impact response rates (length of survey, size of audience, number of segments, etc.) However, based on a pure probabilistic random sample draw, 400 completes will obtain a +/- 5% margin of error whether your audience is 10,000 or 950,000. A recommended industry standard 5% margin of error dictates each statistic you view in the report can range by a total of 5% points (e.g., if overall satisfaction is 84%, you can be confident if you were to conduct the survey again with another 400 customers, 95 out 100 times results for overall satisfaction would yield between 79% and 89%.) However, 400 completes is difficult if not impossible to obtain if your sample size is not considerably greater to pool from.
1
Additional data also provides flexibility in analysis. In aggregate, 400 completes will incur enough reliability for collective analysis but your reliability of data decreases with each cut or segment you analyze. For instance, if you want to compare United States respondents to Asia respondents you might incur sample sizes of 125 and 75 respectively. As a next step you may want to look solely at ABC industry respondents from the Asia market which out of the 75 respondents, could detail a sample size of 13. Making data-driven strategy decisions based on 13 completes does not incur strong reliability but will provide some anecdotal direction.
What Are Some Practical Applications to Increase Response Rates?
Here are 14 ways to increase survey response rates within your company. These recommendations are not mutually exclusive and one often impacts the other. They are broken down by big impacts, medium impacts, and small impacts but even those can be argued. Also note some recommendations are easier to implement than others and each have individual positive impacts on the number of responses your survey will receive. These ground-level incremental efforts should be reviewed when each new survey opportunity arises.
Big Impacts
1) Pre-test the survey – this is conducted both in-house and through a possible pilot test with potential respondents. This pre-testing will identify areas of confusion, potential survey flags which will encourage drop-off, and will also allow the research team to identify any programming issues with the survey before it reaches the masses. After the pre-testing, you also may want to consider a soft-launch where you survey 1 to 2% of your audience to review results and ensure the data you are receiving is what’s needed. Here are 10 benefits to pre-testing your online survey.
2) Subject line matters – the majority of respondents will determine whether or not they will take the survey or even open the email based on the subject line itself. Spend some time thinking through the words used and what will catch the attention and interest of the reader. Ambiguity can be both good and bad (e.g., We Need Your Feedback!) Try to customize and personalize the subject line as much as possible even if it means using separate mail merge invitations for each of your 8 targeted audiences rather than one general email.
2
Big Impacts (Continued)
3) Use multiple channels – some of the best ROI from research efforts come from a round of follow-up phone calls in combination with an online survey. This can be done internally or through a research call center vendor. Calls are placed to remind respondents to take the survey and then a final reminder can be sent following the call (or the following day). Some companies even choose to give the respondent the option to complete by phone if willing. Also consider publicizing the link on social media or using direct mail with a postage paid return envelope. In a digital world, grass-roots efforts through reminders during in-person visits and mail often make a large impact if everyone chips in.
4) Incentives – no initiative will have a greater impact on response rate than rewarding or incentivizing participation in research. Incentives can come in the form of raffles for large gift cards, small nominal payouts to each respondent, or other discounts. When a respondent is considering participating in a survey, they are weighing a few items, most notably their time versus the value they will receive for their time. Using hard figure payouts allows the respondent to create a tangible time versus money payout. Is there a magic number or budget to apply to each project? No, but the conclusion across the market research industry is the more, the better and here's why. Paying $10 per survey will earn a higher response rate than paying out an entry into a raffle to win a $100 gift card.
3
Medium Impacts
5) Make it relevant – if your company has the flexibility consider a pre-notification letter, a headline in your company e-newsletter, or article on your company website to make the intended audience aware of efforts around surveying and why it is important for them (and you). A static page on the website could be created so links in surveys can push traffic to this page. Typically the letter will come from the owner of the company or major stakeholder with a wide organizational impact with a picture and signature. The relevance should discuss the importance of the survey effort(s), how the data will be used, and how direct feedback will benefit each and every customer willing to respond. This type of effort is double-barreled creating both additional awareness and buy-in. You can even choose to share some of the topline results with survey participants after the project is complete.
6) Keep language short and simple – this is implied across all communications (subject line, invitation language, and survey questions). The goal is to make your point as quickly and as clearly as possible and remove the “fluff.” If you can ask a survey question in 10 words, use 10 words, not 15. The simplicity in survey design will not increase open rates but it will decrease drop-off rates once engaged. Place the survey link at or near the top of the email text, this allows users to get right into the survey immediately or read the “fluff” if they choose underneath.
4
Medium Impacts (Continued)
7) Consider the timing of the invitations and reminders – keep track of data and responses during times of the day and days of the week from your audience. Consider using questions in your survey which ask the user what day(s) of the week and what time(s) of the day work best for them to respond to a survey. This can be tackled near the end of the survey or just before the demographics. Although there is no golden rule for when to send invitations, and many sources conflict on the best date(s) and time(s), I've generally seen some research which points to B2B surveys being best sent Monday afternoons or Tuesday mornings. To increase open and response rates for B2B surveys, avoid Monday mornings, Friday afternoons, and weekends. Also consider asking your respondents for their preferred mode(s) of contact for survey efforts as a survey question. Why not right?
8) Format matters – make sure your survey software or your vendor uses device ID codes to change survey format on the fly depending on the type of device being used (desktop, tablet, phone, etc.) The look and feel of the experience needs to be consistent. If any formatting errors occur during the user experience (UX) you will stand a large chance of losing the respondent. Some vendors take the device ID a step further and adapt the number of questions based on device (e.g. if being taken on a phone the user only receives the 3 to 4 core questions, whereas the desktop user will receive the full 15 questions).
9) Make the survey engaging – look to switch up scales, use visuals, ask creative questions, and improve the design and aesthetics of the users’ experience on the survey link. Limit grid questions when possible. Research shows repetitive grid questions (e.g., rate these 10 factors on a scale of 1 to 5 are significant predictors of survey drop-offs.) Consider conjoint analysis if you have a large number of concepts or features to test so not all respondents rate all potential packages but you still obtain reliable data for each.
5
Medium Impacts (Continued)
11) Be clear on confidentiality – this is typically done in the final few sentences of the survey invitation language in the email text. Will responses remain anonymous? Will my answers remain confidential? This is clearly more important in customer satisfaction (CSAT) studies than it would be in more general market or industry surveys. If respondents are providing direct feedback on people or service-levels within an organization, they may not want their personal information divulged. If you are unsure about how to breach this topic, it is always a good suggestion to include a question at the end of the survey which asks the respondent if they would be willing to discuss their survey answers further. In today's world of making research actionable, confidentiality is not king anymore in surveying.
Small Impacts
10) Keep the survey length short and simple – if your online survey takes the respondent more than 3 to 5 minutes, you will experience a higher rate of drop-offs. If your survey must extend beyond 3 to 5 minutes, you should consider offering higher rewards or incentives or using question sequence randomization (e.g., half of your respondents receive questions 1 to 10 and the other half receive questions 11 to 20).
6
12) Offer an unsubscribe link – this would allow the respondent to remove themselves from future research efforts. Before you let them go, ask one final question about why they are dropping out to gain valuable feedback to limit future churn (not enough value, too many surveys, surveys too long, all of the above, etc.) This unfortunately will happen regardless of whether you write great surveys. Nothing you can do about this other than to find out "why" they want out.
Small Impacts (Continued)
13) Determine thresholds – if multiple surveys are taking place across departments within a company and the same respondent runs the risk of receiving several survey invitations from your company over a specific timeframe, you should identify contact thresholds (once every 6 months, once every year, etc.) This step will ensure your respondents are not participating because they feel overwhelmed by the amount of invitations they receive. A central calendar should be created which details company-wide survey efforts and audiences the research used.
Small Short-Term Impact, Large Long-Term Impact
14) Consider developing an in-house panel – to counteract poor response rates, corporations across the globe are immersing themselves in the growth of customer panels. A panel is a group of key stakeholders, customers, and non-customers who agree to participate in future research efforts. They essentially say “contact me when you need me!” These participants opt-into helping out the company on future ad-hoc research requests as well as more formal invitations. It creates buy-in and ownership from participants and often involves incentives each time you reach out to them for feedback (whether it is an in-depth interview (IDI), online focus group, or telephone survey.) Benefits of panels include quick turnarounds for research (time) and easily accessible audience when it is needed (money). This initiative has immense long-term payoffs for a company. Once your panel is large enough, response rates to surveys will detail much higher response rates than any of your efforts placed with non-panel members. So essentially you look to grow your panel each and every day and it becomes an ongoing growth strategy.
