Embed Size (px)
Citation preview
Winning Ads, Logos, and Packaging
Concept AssessmentA How-to Guide to Developing
2
2 Summary
What you’ll learn in this guideConcept Assessment is the process of testing everything from evaluating consumers’ response to a new product idea to altering an existing logo based on consumer attitudes.
We’ll show you how to use surveys to test concepts by taking the guesswork out of your big decisions.
Our guide includes:
• Tips for getting key metrics
• Best practices on writing concept testing surveys
• Real-world examples of how big brands use surveys to make decisions
Real-world examplesAd concept testing: Budweiser®
• Emotional messaging wins over consumers: Overall, the “Lost Puppy” ad received higher scores for Purchase Intent, Watchability, Uniqueness, and Relevance but the “Brewed the Hard Way” ad scored higher in Believability.
Logo concept testing: Airbnb®
• New and improved: The new logo (The Bélo) received higher scores across the board in Appeal, Uniqueness, Relevance and Purchase Intent than the older logos.
Package concept testing: Ben & Jerry’s®
• Old is gold: The previous Cherry Garcia package ranked the highest in all 5 packaging metrics, and was the favorite packaging of two-thirds of respondents.
Created by and for the use of SurveyMonkey Audience for marketing purposes. This study was not commissioned or endorsed by Budweiser®, Airbnb®, Ben & Jerry’s® or any other companies measured in the study
3
Table of Contents 3
Overview
Measuring Success
How to: 3 Real-World Examples
4
6
8
Ben & Jerry’s Study: Packaging Concept Test
Budweiser Study: Ad Concept Test
Airbnb Study: Logo Concept Test
10 13 16
4
Overview
5
5 5 Benefits of Testing Concepts with Surveys
1. Test many concepts quickly and easilyA survey allows you to present a wide range of ideas to your target market, and get quantitative feedback fast.
2. Filter out the bad ideasLet your target market decide what works and what falls flat by asking them directly to investigate what customers want.
3. Refine your good ideasIf you test a bunch of concepts through a survey, you can quickly see which ones are rising to the top.
4. Test with diverse audiencesIt’s much easier to use surveys than expensive focus groups to reach different demographic groups in your target market and segment their responses.
5. Save moneyAfter you’ve launched your first concept testing survey, take a look at the results and see if any ideas tanked. Now imagine having spent months and thousands of dollars developing those ideas.
6
6 How to Determine Concept SuccessThe quickest way to determine a winning concept is to utilize a scoring model. This is an example scorecard that incorporates several key success metrics depending on the type of concept you’re testing and how you define a successful concept.
A B
Attributes Tested
Purchase Intent Statistical Significance
% %
Appeal Statistical Significance
% %
Uniqueness Statistical Significance
% %
Relevance Statistical Significance
% %
Average Score % %
*Z-test scores at 95% confidence interval
Created by SurveyMonkey Audience
Top 2 Box: the percentage of respondents who answered the two most positive answer options
Step 1: Define your key success metrics
Step 5: Determine which concepts performed better by using “stat testing,” a calculation that shows if two numbers or scores have a significant, or big enough, difference to show if they truly resonate with consumers more than the others; A and B designate significant differences between concepts
Step 3: Use the top 2 answer choices for each question to more easily compare how each concept performed
Step 2: Set the number of metrics depending on the concept you’re testing and how you define success
Step 4: Compare the average scores of each concept to determine which one performed best on the success metrics you chose
7
7 Key Success Metrics
Here are several key metrics you can incorporate into a concept test:
Whether your concept motivates viewers to buy
If your concept is of high quality according to respondents
The overall appeal of your concept is a strong indicator of purchase intent
How well your messaging comes through
A gauge of how well your target audience remembers your message or brand
If your packaging can set itself apart from the others
How different one concept is from the others
How trustworthy your concept is perceived to be
A way to determine if your concept resonates with respondents
How likely your ad will be watched over and over
Purchase Intent:
Perception of Quality:
Appeal:
Message Clarity:
Recall:
Distinctive Packaging:
Uniqueness:
Believability:
Relevance:
Watchability:
8
3 Real-World Concept Assessment Examples
9
9 Methodology for 3 Studies
The survey field dates were as follows:
• Airbnb: March 16 – 17 2015
• Ben & Jerry’s: March 16 – 17 2015
• Budweiser: March 19 – 20 2015
SurveyMonkey Audience fielded 3 Concept Assessment studies to demonstrate how to successfully test ads, logos, and packaging.
The surveys were fielded through the SurveyMonkey panel. Respondents represented the general U.S. population. A total sample size of 1,039 was used for this study; survey sample sizes were as follows:
• Airbnb: 423
• Ben & Jerry’s: 316
• Budweiser: 300
What?
How?
When?
10
Budweiser Study:Ad Concept Test
11
11 Budweiser Ad Test Summary
Background: The Superbowl is known to host some of the most engaging, effective, and expensive ads on television every year.
Goal: Put two in-field ads head to head to see which one performs better.
Key Insights:
Emotional messaging scores high in Purchase Intent: the “Lost Puppy” ad provoked higher Purchase Intent for Budweiser beer than the “Brewed the Hard Way” ad.
A lost puppy wins over consumers: The emotional ad piqued the interest of respondents who were significantly more likely to rewatch the "Lost Puppy," despite the ad having a similar Relevance score as "Brewed the Hard Way."
See it to believe it: "Brewed the Hard Way" has a more informative, product-related spin on Budweiser beer and received a significantly higher score in Believability.
1 2 3
12
12 Budweiser Scorecard
Concept Test Scorecard (Top 2 Box Scores)
A B
Attributes Tested n=300 n=300
Purchase Intent Statistical Significance
Top 2 Box = Definitely would buy + Probably would buy 36% (B) 31%
Watchability Statistical Significance
Top 2 Box = Strongly agree + Somewhat agree 78% (B) 32%
Uniqueness Statistical Significance
Top 2 Box = Extremely different + Very different 61% (B) 32%
Relevance Statistical Significance
Top 2 Box = Extremely relevant + Very relevant 35% 28%
Believability Statistical Significance
Top 2 Box = Extremely believable + Very believable 32% 49% (A)
Average Score 48% 34%
*Z-test scores at 95% confidence interval
“Lost Puppy” “Brewed the Hard Way”
13
Airbnb Study:Logo Concept Test
14
14 Airbnb Logo Test Summary
Background: Airbnb went through a complete rebranding in late 2014 which included a very controversial reception of their new logo.
Goal: Understand how this rebranded logo is perceived compared to the two older ones.
Key insights:
1 2 3
A unique logo goes hand-in-hand with an emerging brand: “The Bélo,” the new Airbnb logo, is perceived as a very unique logo when compared to the previous ones.
Improved relevance and appeal: The new logo resonated well within the respondent base and was seen as more appealing and more relevant compared to the previous ones.
Persuasion is higher: Higher scores in Uniqueness, Relevance and Appeal translates into a stronger Purchase Intent score for Airbnb with the new logo.
15
15 Airbnb Scorecard
Concept Test Scorecard (Top 2 Box Scores) Survey Results as of: March 17, 2015
Blue and Purple Blue and White The Bélo
A B C
Attributes Tested n=423 n=423 n=423
Purchase Intent Statistical Significance
Top 2 Box = Extremely likely + Very likely 9% 9% 12%
Appeal Statistical Significance
Top 2 Box = Extremely appealing + Very appealing 15% 17% 20%
Uniqueness Statistical Significance
Top 2 Box = Extremely unique + Very unique 13% 17% (A) 30% (AB)
Relevance Statistical Significance
Top 2 Box = Extremely relevant + Very relevant 15% 13% 17%
Average Score % Ranked #1
13% 33%
14% 28%
20% 39%
*Z-test scores at 95% confidence interval
16
Ben & Jerry’s Study: Packaging Concept Test
17
17 Ben & Jerry’s Packaging Test Summary
Background: With high brand awareness and years on the market, Ben & Jerry’s has made few changes to its packaging while its distribution channels have multiplied. This could be a case study on how not to change what works.
Goal: To learn what ice cream buyers felt about the design differences between two ice cream packages. The catch? We didn’t tell them which package was the “new” one.
Key insights:
1 2 3
Old is gold: The previous Cherry Garcia package ranked the highest in all five packaging metrics, and was the favorite packaging of 65% of respondents.
Communication is key: When we asked respondents “How clearly does the package communicate the product or flavor inside?” the old package ranked significantly higher than the new package, suggesting that the visual elements on each package had varying impacts upon respondents.
Equally unique: Despite design differences in the packaging, respondents agreed that both packages showcased unique or distinct designs, compared to other ice cream brands in the market.
18
18 Ben & Jerry’s ScorecardConcept Test Scorecard (Top 2 Box Scores)
A B
Attributes Tested n=161 n=155
Appeal Statistical Significance
Top 2 Box = Extremely appealing + Very appealing 62% (B) 51%
Purchase Intent Statistical Significance
Top 2 Box = Extremely likely + Very likely 30% 25%
Distinctive Packaging Statistical Significance
Top 2 Box = Extremely distinctive + Very distinctive 69% 67%
Perception of Quality Statistical Significance
Top 2 Box = Extremely high + Very high 70% (B) 57%
Message Clarity Statistical Significance
Top 2 Box = Extremely clearly + Very clearly 87% (B) 69%
Average Score % Ranked #1
64% 65%
54% 35%
*Z-test scores at 95% confidence interval
Cherry Garcia (Old) Cherry Garcia (New)
19
Taking It to the Next Level
20
20 Going Beyond Your First TestConcept testing goes beyond asking respondents to rank and rate concepts against one another.
Take it a step further: • Ask respondents open-ended
questions to get the “Why?” behind their answers
• Track your performance over time by setting up recurring surveys
Need some help selecting metrics? We can help. • SurveyMonkey Audience can
provide you a list of all the available key metrics you can test and get respondents to rate concepts on
The possibilities are endless: • Package testing, logo testing, name
testing, claims testing...the list goes on
• Concept testing spans a variety of categories—all you have to do is provide us the stimuli, and we can make that concept assessment happen
21
Bring your best ideas to market with concept testing
Learn more about testing your ads, logos, and packaging by contacting SurveyMonkey Audience today
Contact Us to Get Started
© 2015 SurveyMonkey. SurveyMonkey, the SurveyMonkey logo, SurveyMonkey Audience and the SurveyMonkey Audience logo are trademarks of SurveyMonkey Inc. and its affiliates. Other company and product names may be trademarks of their respective owners.
