Upload
marketingexperiments
View
2.250
Download
1
Embed Size (px)
DESCRIPTION
By the end of this year, MarketingExperiments will have run approximately 260 controlled experiments. That equates to around 50,000 total hours of research. With the sheer volume of work done this year, it’s almost impossible for marketers in the field to keep up. In our next Web clinic, we’ll walk through the most important discoveries of the year so you can learn the progress MECLABS analysts and scientists have made. You’ll discover actual research-grounded answers to the following questions: • When is the best time to send an email? • How should I design my website navigation? • Can a brand really make an impact on conversion? • Is my page missing critical copy? • And more … You’ll also receive access to the broadest array of researchers from our lab so far this year. They’ll be talking through the experiments and answering your questions live from the MarketingExperiments broadcast studio.
Citation preview
The Top 5 Marketing Discoveries in 2013The last 50,000+ hours of research distilled into 60 minutes
We’re sharing on Twitter!#WebClinic
Today’s speakers
Austin McCraw
Senior Editorial Analyst
MECLABS
Jon Powell
Senior Manager Research and Strategy
MECLABS
Pamela Markey
Senior Director
Marketing MECLABS
Adam Lapp
Director
Services Operations
MECLABS
Ben Filip
Senior Manager Data Sciences
MECLABS
Research sampling published in 2013275%Conversion
109%Conversion 188%
Conversion
48%Conversion
220%Conversion
155%Conversion
97%Conversion
17%Clickthrough
43%Conversion
36%Conversion
101%Conversion
87%Conversion
68%Revenue/Visit
C = 4m + 3v + 2(i-f) - 2a ©
ec < op < ct < lp ©
eme = rv(of+ i) – (f + a) ©
fSC = lT + dT©
Opr > Oprn > Ocnn ©
Today, we are going to distill all of this research into a 60-minute walkthrough of the top five discoveries in 2013.
Discovery #1:
There are Five Consistent Factors that Impact Customer Response to Email Timing
Experiment: Background
Background: A large financial institution offering a financial service requiring an application to consumers.
Goal: To increase the amount of completed applications.
Primary Research Question: Of the send times tested, which time will result in the highest rate of completed applications to delivered emails?
Approach: A/B multifactor sequential test
Experiment ID: TP2087Record Location: MarketingExperiments Research LibraryResearch Partner: Protected
Research Notes:
Experiment: Design
The test had a total 14 treatment paths – each path had the same subject line and email.
The treatment values included sending two emails each day of the week, Monday through Sunday, at 3 a.m. and 3 p.m.
14 Total Email Sends During Week of Test
Monday
AM Email
PM Email
Tuesday
AM Email
PM Email
Wednesday
AM Email
PM Email
Thursday
AM Email
PM Email
Friday
AM Email
PM Email
Saturday
AM Email
PM Email
Sunday
AM Email
PM Email
2 4 6 8 10 12 14
1 3 5 7 9 11 13
Which Time of Day Will Perform Best?3 a.m. or 3 p.m.?
?
Results
Clickthrough Rate
3:00 a.m. 1.076%3:00 p.m. 1.220%
Percent Relative Change 13.5%
! What You Need to Understand: Recipients were 13.5% more likely to click in an email sent at 3 p.m.
increase in clickthrough Sending at 3 p.m. increased clickthrough rate by at least 13.5%.
13.5%
Which Day of the Week Will Perform Best?Mon., Tues., Wed., Thurs., Fri., Sat., Sun.?
Experiment: Clickthrough rate by day
Mon Tues Wed Thurs Fri Sat Sun
Clickthrough Rate By Day
Treatments (in order of performance)
ClickthroughPercent Relative
ChangeLOC
Tuesday 10.47% - -
Monday 10.90% 4.1% 72%
Thursday 11.01% 5.1% 82%
Friday 11.59% 10.7% 99%
Wednesday 11.64% 11.2% 99%
Saturday 11.84% 13.1% 99%
Sunday 12.90% 23.2% 99%
! What You Need to Understand: Recipients were 23.2% more likely to click in an email sent on Sunday than one sent on Tuesday. Sunday also significantly out-performed every other day of the week.
increase in clickthrough Sending on Sunday increased clickthrough rate by 23.2% over Tuesday.
23.2%
Experiment: Clickthrough rate by day
Experiment: Interpretation
Possible Reasons for Higher Clickthrough on Sunday
• Less distractions on Sunday
• Advent of mobile attaches recipients to email 24/7
• Personal nature of financial product causes higher clickthrough outside of work
Sunday Reader
Experiment: Interpretation
Possible Reasons for Lower Clickthrough During Week:
• Higher level of distraction
• Customers are in “work” mode and less likely to be interested in financial application
Weekday Reader
What we discoveredKey Principles
F1. There is no “one-size-fits-all” time or frequency for an email send.
2. However, the universal goal of email timing is to synchronize your email’s delivery with the cognitive psychology of the customer’s purchase cycle.
3. There are five consistent factors from campaign to campaign that will impact your customers’ purchase cycle:
1. Decision cycle – (How often a decision is required)2. Utility – (How useful the email is perceived)3. Relevance – (How relevant the email is externally and internally)4. Nature of product – (How the product affects the customer emotionally)5. Expectations – (How the email matches customer expectations)
View the Full Clinic Here:
MarketingExperiments.com/EmailTiming
Discovery #2:
Brand Does Not Make a Promise, It Creates an Expectation
Experiment: Background
Background: One of the largest metropolitan print news sources in the United States.
Goal: To increase the number of home delivery subscriptions.
Research Question: Which offer page will result in the highest subscription rate?
Test Design: A/B variable cluster test
Experiment ID: TP1651Record Location: MECLABS Research LibraryResearch Partner: The Boston Globe
Research Notes:
Experiment: Control
• The goal of the original page was to get people into the subscription process.
• The original page used a template CMS structure that did very little to leverage the brand of The Boston Globe.
Experiment: Control
Offer Page
Step 1 Step 2 Step 3 Step 4
• The unbranded template is used throughout the entire conversion process.
Experiment: Treatment
• The treatment slightly adjusts the CMS template to emphasize The Boston Globe brand.
Experiment: Treatment
• Boston Globe branding is made prominent throughout the entire conversion process.
Experiment: Side-by-side
Will emphasizing a well-known brand in the online subscription process generate more response?
Experiment: Results
Clickthrough Rate
Control (Non-Branded) 1.32%Treatment (Branded) 1.86%
Percent Relative Change 40.3%
! What You Need to Understand: By simply emphasizing the well-known brand of The Boston Globe, the treatment subscription path generated 40.3% more subscriptions than the generically branded control.
increase in total subscribersThe branded subscription path increased the rate of subscriptions by 40.3%
40%
What we discovered
Brand is the aggregate experience of the value proposition
Key Definition
What we discovered
Key PrinciplesF
1. Brand represents the sum total of experiences in the market place, particularly those that connect to you.
2. Brand exists in the mind. It represents a form of mental shorthand. It stands for a decision collective
and represents the default choice. As such, it implies expectation.
Brand does not make a promise; it creates an expectation. The strength of the brand is derived not from declaration, but through expectation.
3.
View the Full Clinic Here:
MarketingExperiments.com/DoesBrandMatter
Discovery #3:
Usability Does Not Equal “Buyability”; All Usability Claims Must Be Tested
Experiment: Background
Background: Italian e-commerce website offering cosmetics. The researchers were focusing on testing different approaches to the “body” category page.
Goal: To increase the rate of conversion.
Primary Research Question: Which page will generate the highest rate of conversion?
Approach: A/B variable cluster test
Experiment ID: TP1283Record Location: MECLABS Research LibraryResearch Partner: Protected
Research Notes:
Experiment: Control
• The control listed all of the main categories of “body” products:
• Deodorants• Moisturizers• Toner• Skin• Hair remover• Feet• Hands• Mouth• Scrubs and specials• Accessories• Combination offers
• Products are all listed below by category as selected
Control – Category List
Experiment: Control
Is the category list at the top of the page the most user-friendly way to present the information?
Control – Category List
T1 – Configurator
Experiment: Treatment 1
• Treatment 1 seeks to make the page easier to use by adding an interactive configurator that enables the visitor to customize the products that show up below.
• By Category• By Objective• By Product Line
T2 – Visual Categories
Experiment: Treatment 2
• Treatment 2 seeks to make the page easier to use by removing the category links and simply featuring the main categories with images.
Experiment: Treatment 3
• Treatment 3 is a radical approach that seeks to make the process easier by removing the “body” category page altogether, enabling the visitor to choose their category within the navigation of the homepage.
T3 – Navigation Links (text)
T4 – Navigation Links (visual)
Experiment: Treatment 4
• Treatment 4 is similar to Treatment 3, only it integrates a more visual approach to the categories within the navigation.
Experiment: Test your intuitionControl – Category List
T1 – Configurator T2 – Visual Categories
T3 – Navigation Links (text) T4 – Navigation Links (visual)
Results
increase in total conversions Treatment 1 category page increased conversion rate by 20%.
20%
Treatments Clickthrough Percent Relative Change
Control – Category List 1.04% -
T1 – Configurator 1.25% 20%
T2 – Visual Categories 1.10% 6%
T3 – Navigation Links (text) 1.10% 5%
T4 – Navigation Links (visual) 1.10% 5%
Experiment: Results
Why did the configurator beat all of the other methods? Was it the usability? If so, what made it more usable than the other methods?
Control – Category List
T1 – Configurator
What we discovered
1. Focusing on usability can hinder an effective approach to optimization. It often undermines the micro-yes
approach and can ultimately hinder results.
2. An overemphasis on usability confuses the means with the end. The goal of our website is not usability,
but rather “buyability.”
3. Usability, when viewed properly, can be a valuable tool for helping marketers identify hidden psychological
costs in their conversion processes.
4. All usability claims must be tested.
Key PrinciplesF
View the Full Clinic Here:
MarketingExperiments.com/UsabilityMyth
Discovery #4:
Letter-Style Emails are More Effective than Promotional Emails
Experiment: Background
Background: A large international media company focusing on increasing subscription rates.
Goal: To increase the number of conversions based on the value proposition conveyed through the email.
Primary Research Question: Which email will generate the highest conversion rate?
Approach: A/B multifactor split test
Experiment ID: TP2137Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Research Notes:
Experiment: Control
CONTROL: Promotional-style email
• Uses popular design principles to create balance and hierarchy on the page.
• Heavy use of images and graphics to catch the reader’s attention.
• Multiple call-to-action buttons for increased points of entry.
Subject Line: Open this now for Special Savings
TREATMENT: Letter-style email
• Designed to look and feel more like a personal letter.
• Limited use of graphics and images.
• One call-to-action button.
Experiment: Treatment Subject Line: Get Unlimited Access to [Product] with Home Delivery
Subject Line: Open this now for Special Savings
Experiment: Side-by-sideControl
VS.
Subject Line: Get Unlimited Access to [Product] with Home Delivery
Treatment
Experiment: Results
Conversion Rate
Control – Standard Email 0.04%Treatment – Letter-Style Email 0.12%Relative Difference 181%
! What You Need to Understand: By limiting the amount of graphics, and focusing on engaging the customer in a conversation, the treatment outperformed the control by 181%.
increase in conversion The overall conversion rate increased 181% due to a clearly stated value proposition.
181%
Why did the letter-style email win?
Subject Line: Get Unlimited Access to [Product] with Home Delivery
Why did the letter-style win?
1. An email message is not a monologue; it is a dialog. People don’t buy from emails; people buy from people.
2. If the marketer can learn to participate with the prospect’s conversation, they can guide it (with messaging) toward a satisfactory conclusion (the purchase).
3. Therefore, effective email messaging requires one often overlooked skill on the part of the marketer: empathy.
F Key Learnings
What we discovered
View the Full Clinic Here:
MarketingExperiments.com/LetterEmails
• Selfishness, if a benign version, is the primary driver of sales velocity. It is the selfishness of the prospect which empowers the transaction.
• The concept, selfishness, has a negative connotation. But this can be unfortunate – how can a self be faulted for being self(ish)? In one sense, selfishness is essence(tial).
• Empathy enables the marketer to identify with the market and experience its “selfishness.” Empathy is the marketer’s intuition.
Empathy: For the marketer, empathy is their ability to discern – through listening and hearing – the ontology (nature or being) of the customer.
What we discovered
Discovery #5:
Copy Should Establish the Problem before Presenting the Solution
Background: A medium-sized company selling a single auto repair product.
Goal: To increase the number of purchases.
Primary Research Question: Which landing page will generate the highest conversion rate?
Approach: A/B multifactor split test
Experiment ID: TP1700Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Research Notes:
Experiment: Background
Experiment: Control
Logo
Brand
Experiment: Treatment
Logo
Brand
Experiment: Side-by-sideControl
Treatment
Experiment: Results
Conversion Rate
Control 1.33%Treatment 1.81%Relative Difference 36%
! What You Need to Understand: By clarifying the problem before presenting the solution, the treatment generated a 36% higher purchase rate than the control.
increase in purchasesProduct purchases increased by 36% over the control.
36%
What we discovered
1. Simply identifying a customer need does not inspire the need to act. The problem must be intensified so that it is properly felt by customers.
2. To inspire action, your copy must sufficiently transform a customer need into a customer want. This can be achieved by using one or more of three problem intensifiers:
• Relevance: the degree to which an offer is connected to a recipient’s situational motivations.
• Importance: the degree to which an offer is essential to a recipient’s livelihood.
• Urgency: the degree of immediacy associated with an offer.
FKey Principles
View the Full Clinic Here:
MarketingExperiments.com/MissingCopy
Upcoming Research in 2014
EARLY BIRD SAVINGS
Experiment: Background
Background: A large news media organization trying to determine whether they should invest in responsive mobile design
Goal: To increase the number of free trial sign-ups.
Primary Research Question: Which design will generate the highest rate of free trial sign-ups: responsive or unresponsive?
Approach: A/B multifactor split test
Experiment ID: TP1933Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Research Notes:
Experiment: Control (Unresponsive)Desktop
Mobile
Experiment: Treatment (Responsive)
Desktop iPhone
Experiment: Side-by-sideUnresponsive Responsive
Does Responsive Mobile Design Actually Convert Better?
Coming up in January 2014
Summary: Putting it all together
Discovery #1:There are Five Consistent Factors that Impact Customer Response to Email Timing
Discovery #2:Brand Does Not Make a Promise, It Creates an Expectation
Discovery #3:Usability Does Not Equal “Buyability”; All Usability Claims Must Be Tested
Discovery #4:Letter-Style Emails are More Effective than Promotional Emails
Discovery #5:Copy Should Establish the Problem before Presenting the Solution