LCMC Session: Email and Web Conversion Success

The Formula for Email and Web Conversion SuccessJon PowellMECLABS

MarketingSherpa | MarketingExperiments

Top Obstacles in Email

Staffing, expertise

Creating consistent, relevant content

Merging Email Data

1 billion+ emailmessages

400 uniquerules

Evaluates performance from: • 1 billion+ sent and

tracked email messages

• Over 400 unique rules

An ExperimentEmail messaging effectiveness

Background: Large financial institution offering mortgages to consumers.

Goal: To determine which claim of value will generate the most response.

Primary research question: Which subject line will produce the greatest open rate and click rate?

Approach: A/B/C single-factorial split

Experiment ID: (protected)Location: MarketingExperiments Research LibraryTest Protocol Number: TP2092�

Test Scenario 1

Test Protocol ID: TP2092

Looking for a lower mortgage rate? You’re not alone.

Version A

Version B

Version C

Discover the advantages of [Brand Name] mortgages

Own your home sooner with free bi-weekly payments

SL Rating

81

78

63

Test Scenario 1

Subject Line Tool Rating

Open Rate

Rel. Diff.

A Looking for a lower mortgage rate? You’re not alone. 63/100 18.08% -

B Discover the advantages of [Brand Name] mortgages 78/100 19.80% 9.5%

C Own your home sooner with free bi-weekly payments 81/100 20.99% 16.0%

Accurate PredictionVersion B & C generated a statistically significant improvement over A


Open Rate

Rel. Diff.

A Looking for a lower mortgage rate? You’re not alone. 63/100 18.08% -

B Discover the advantages of [Brand Name] mortgages 78/100 19.80% 9.5%

C Own your home sooner with free bi-weekly payments 81/100 20.99% 16.0%

Click Rate

Rel. Diff.

0.61% -

0.57% -7.4%

0.68% 11.3%

Accurate PredictionVersion B & C generated a statistically significant improvement over A Mostly Accurate PredictionVersion C still generated a statistically significant improvement over A

Background: A well-known news publication

Goal: To increase subscriptions from former subscribers who have been absent between one and two years

Primary research question: Which subject line will produce the greatest open rate?

Approach: A/B/C single-factorial split

Experiment ID: (protected)Location: MarketingExperiments Research LibraryTest Protocol Number: TP2078�

Test Scenario 2

Test Scenario 2Test Protocol ID: TP2078

Choose from 2 ways to save big on the NEW [PublicationName].com

Version A

Version B

Save 50% on your choice of two new subscription options

Version C

Home Delivery comes with FREE access to [PublicationName].com

SL Rating

56

59

50


Open Rate Rel. Diff.

A Choose from 2 ways to save big on the NEW [PublicationName].com 56/100 9.43% 50.2%

B Save 50% on your choice of two new subscription options 59/100 7.01% 11.7%

C Home Delivery comes with FREE access to [PublicationName].com 50/100 6.28% --

Inaccurate PredictionWhile Version A generated a statistically significant response, Version B did not


Open Rate Rel. Diff.

A Choose from 2 ways to save big on the NEW [PublicationName].com 56/100 9.43% 50.2%

B Save 50% on your choice of two new subscription options 59/100 7.01% 11.7%

C Home Delivery comes with FREE access to [PublicationName].com 50/100 6.28% --

Click Rate Rel Diff.

1.48% 86%

0.76% -5.3%

0.80% --

Inaccurate PredictionWhile Version A generated a statistically significant response, Version B did not

Test Scenario 3Test Protocol ID: TP2076

Target MDs with Surveys and PanelsVersion A

Version B

Reach 120,000 Physicians through Social Media

SL Rating

94

76


Open Rate Rel Diff.

A Target MDs with Surveys and Panels 94/100 21.30% --

B Reach 120,000 Physicians through Social Media 76/100 22.92% 7.6%

Inaccurate PredictionVersion B generated a statistically significant higher response than A


Open Rate Rel Diff.

A Target MDs with Surveys and Panels 94/100 21.30% --

B Reach 120,000 Physicians through Social Media 76/100 22.92% 7.6%

Inaccurate PredictionVersion B generated a statistically significant higher response than A

Experiment: Results

The tool predicts a winner:

31.57% of the time

After rating hundreds of subject lines from our library of statistically significant experiments…

A coin predicts a winner:

50% of the time

25¢ Subject Line Rating ToolCheck Yours Now!

The tool predicts a winner:

31.57% of the time

So what went wrong with the tool?Decisions based on violations:• Does not contain a sense of urgency• Mobile subject line is over 35 characters• Length is over 50 characters• Contains any word that has more than 8

characters• Includes one exclamation mark• Use of a word in all caps• Includes the word “You” or “Your”• Contains comma• Starts with Save• Use of $• Contains % off

So what went wrong with the tool?Decisions based on violations:• Does not contain a sense of urgency• Mobile subject line is over 35 characters• Length is over 50 characters• Contains any word that has more than 8

characters• Includes one exclamation mark• Use of a word in all caps• Includes the word “You” or “Your”• Contains comma• Starts with Save• Use of $• Contains % off

Computers are great with computer logic;

But not so great withhuman logic

• Computer logic draws conclusions from algorithms, rules, switches and settings

• Human logic draws conclusions from context, emotions, desires and belonging.

The computer approach

The computer approachIn marketing…

We tend to rely on rule sets• Emphasize the benefits• Avoid long copy, use bullets• Always include a call to action• Keep it above the fold• Emphasize hero shots• Minimize compliance text• Use tabs for additional info

The ResearchMarketingExperiments was the first Internet-based research lab to conduct experiments in optimizing the conversion rate of sales and marketing processes.

1992 Preliminary Research Begins

1997 Research Program Established

2001 First Research Report Published

2002 Testing of Research Partnership Model Begins

2003 Offer Response Optimization Theory Validated

2006 Patent Filings for Research Findings (10 Heuristics)

The ResearchMarketingExperiments was the first Internet-based research lab to conduct experiments in optimizing the conversion rate of sales and marketing processes.

1992 Preliminary Research Begins

1997 Research Program Established

2001 First Research Report Published

2002 Testing of Research Partnership Model Begins

2003 Offer Response Optimization Theory Validated

2006 Patent Filings for Research Findings (10 Heuristics)

People judge emails, websites in the same way they judge people

It’s not the change on the page;It’s the change in the mind

The Result

eme = email messaging effectiveness index rv = relevance to the consumer of = offer to act upon i = incentive to take action f = friction elements of process a = anxiety elements of process

eme = rv(of + i) – (f + a) ©

Email Messaging Optimization Heuristic

eme = rv(of+i) – (f+a)The Result

Perceived Cost

rv ofi

fa

Perceived Value

eme = rv(of + i) – (f + a) ©

The Result eme = rv(of+i) – (f+a)

Relevance (rv)Increasing Perceived Value

Relevance

Relevance -- The compatibility of the email message to the recipient’s motivations.

eme = rv(of + i) – (f + a) ©

eme = rv(of+i) – (f+a)RelevanceTo ThisFrom This

Internal motivations• Personal interests • Demographics• Shopping habits• Personality• Communication styles• Level of engagement

External events• Environmental changes,

realities• Holidays • News events• Competitive initiatives

F Key Principles

eme = rv(of+i) – (f+a)Relevance

Experiment ID: Pier 1 ImportsLocation: MarketingSherpa Research LibraryTest Protocol Number: A-TP1002�

Experiment

Background: B2C home products company with a significant online and retail presence

Goal: To increase the click-through rate from the email to the landing page

Primary research question: Which email design will generate the most clicks?

Approach: A/B/C/D split test (variable cluster)

eme = rv(of+i) – (f+a)


SampleExperiment

Team A

Team B

Team C

Experiment

Team A Team B Team C

Sample


Email Designs Click Rate Rel. Diff.

Sample (client) 36.70% -

Team A Design 17.68% -51.83%Team B Design 29.91% -18.50%Team C Design 24.07% -34.41%

18-52% Decrease in Click-through Each design achieved a statistically significant click-rate decrease

Click-through rate comparisonExperiment eme = rv(of+i) – (f+a)

Team A Team BTeam C Client

17.68% 24.07% 29.91% 36.70%

Experiment eme = rv(of+i) – (f+a)

Team A Team BTeam C Client

17.68% 24.07% 29.91% 36.70%

See a progression?

Offer (of)Increasing Perceived Value

Offer

Offer -- The value you promise in exchange for an action (i.e. a click, a transaction)

eme = rv(of + i) – (f + a) ©

What really is the offer in email?• You may think that the offer is the product that you want

to sell.

• In reality it is the additional information they receive in exchange for a click.

• What is the most effective way to get a click? Is it to give them an entire product pitch? Or just a part?

Offer eme = rv(of+i) – (f+a)


Experiment ID: (Protected)Location: MarketingExperiments Research LibraryTest Protocol Number: TP2801�

Experiment

Background: A physician-only social network that allows medical product companies to conduct first-hand research on potential purchasers.

Goal: To increase the number of leads from a rented trade publication list.

Primary research question: Which email design will generate the most opens, clicks?

Approach: Multi-factorial sequential test on a trade publication list


This email assumes that readers understand the platform’s audience and thus focuses on platform uses

It also assumes intent to purchase with multiple “Get Started” calls to action, one as early as the second sentence


Control

Get started with a free30-minute demo

Get started with a free 30-minute demo

PROTECTED

This version focuses less on platform and more on its users

The primary call-to-action was written to synchronize with their state of interest

TreatmentPROTECTED

Control Treatment


104% Increase in clicks Version B achieved a statistically significant increase in click-rate

Email Click Rate Rel. Diff.

Control 1.55% --

Treatment 3.16% 104%

• In the body of the email, it is important to distinguish the difference between the product offer and the click-through offer.


F Key Principles

• In the body of the email, it is important to distinguish the difference between the product offer and the click-through offer.

• Your communication of value starts with your subject line and should progress, like a conversation, throughout the entire conversion process


F Key Principles


Target MDs with Surveys and Panels

Reach 120,000 Physicians through Social Media

Not this…

But this…

27.6%Increase in clicks



From This To This

Incentive (i)Increasing Perceived Value

Incentive – An additional element introduced to achieve a desired action.

Incentive

eme = rv(of + i) – (f + a) ©


• The purpose of an incentive is to increase the probability of immediate action by tipping the balance of emotional

Incentive

F Key Principles

conflict inside the person

We typically think of…• Discounts (20% off, buy-one-get-one-free)

• Order/Delivery (Free shipping, accelerated shipping, instant download

• Add-ons (subscription extensions, warrantee extensions)

• Content (Whitepapers, books, articles, reports , e-books)

• Accompanying product Incentives • Free accessories (carrying case)• Complementary products (memory card)

eme = rv(of+i) – (f+a)Incentive

Also consider including…

• Instilled Deadlines

• Standard promotions

• Implied exclusivity

• Contests

• Limited Availability

24 hours only: Free shipping on…

This is the last day to enter for your chance to…

Only 25 seats left to see register for...

48-hour member exclusive: new fall styles…



• The purpose of an incentive is to increase the probability of immediate action

F Key Principles

conflict inside the personby tipping the balance of emotional

• In email messaging, it can also include the use of artificial urgency, in which the immediacy associated with the offer is controlled by the marketer.

Friction (f)Decreasing Perceived Cost

Friction – Psychological resistance to a given element anywhere in the email process

Friction

eme = rv(of + i) – (f + a) ©

• Two common elements that create friction in the process are:

• Length• Difficulty

• The objective is to minimize, not eliminate friction. If you eliminate all friction, you eliminate “the sale.”

eme = rv(of+i) – (f+a)Friction

F Key Principles


Not this

But This


Length1

eme = rv(of+i) – (f+a)Friction

Not this But This

52.5%Increase in app starts


Length1

eme = rv(of+i) – (f+a)FrictionTest Protocol ID: TP2081

But this (1 Option)Not this (6 Options)

46%Increase in purchases

Difficulty2

20%Increase in clicks


10%Increase in clicks

Not this But This

Difficulty2


What else contributed to differences in performance?

eme = rv(of+i) – (f+a)FrictionTest Protocol ID: A-TP1002

Not This But This

What else contributed to differences in performance?

eme = rv(of+i) – (f+a)FrictionTest Protocol ID: A-TP1002

Not This But This

Anxiety (a)Decreasing Perceived Cost

Anxiety – Psychological concern stimulated by a given (or unaddressed) element in the email process

Anxiety

eme = rv(of + i) – (f + a) ©

eme = rv(of+i) – (f+a)AnxietyThey’re just hard water stains…

• Anxiety is just as lethal to conversion as friction.

• Try and mitigate anxiety rationally and you are likely to fail…

• Though created from legitimate concern, its impact in the mind is often disproportionate to the reality of risk.

• While website visitors are skeptical and cautious, the natural level of concern is even greater for email offer recipients.

Anxiety

F Key Principles



Background: A large event management online software provider.

Goal: To increase total lead inquiries (phone calls + form sign-ups) from visitors who abandoned the free trial sign-up process.

Primary research question: Which email tone will result in a higher rate of lead inquiries?

Approach: A/B single factorial split


ExperimentSubject Line: Your Free RegOnline Access Dear [Name], You’re just one step away from getting FREE access to RegOnline, our award winning Event Registration and Management Software. Quickly make an event website, try our event marketing tools, build a registration form template or even generate custom name badges. Click here to finish your profile and get started. Your personal profile is kept secure and we promise NEVER to sell or misuse your information. Have questions? Call us direct at 1-800-XXX-XXXX -- we can even set up your free access over the phone. Sincerely, Jon PowellCustomer Service RepresentativeRegOnlineDirect: 1-800-XXX-XXXX ext. [email protected]

• The control email uses a typical sales tone of a direct-response email.

Control

Subject Line: Your Free RegOnline Access Hi [First], I noticed that you started the process of getting free access to RegOnline but weren’t able to finish. Are you concerned about giving out your phone number? Are you worried about high pressure sales tactics or mandatory contracts? We believe our product sells itself, so we’re just here to provide you with whatever assistance you need in getting your event up and running - in whatever way works best for you. We promise NEVER to sell or misuse your information. Call me direct at 1-800-XXX-XXXX and I can help get you rolling. If you’d rather just try again online, use this link instead. Thank you in advance for your trust! Sincerely, [Name of Representative]Customer Service RepresentativeRegOnlineDirect: 1-800-XXX-XXXX ext. XXXX[name]@RegOnline.com

• The treatment directly addresses anxiety, taking on a categorical shift in tone – from a direct-response tone to a customer service tone.

Treatment

Direct Response Tone

Control

Customer Service Tone

Treatment


349% Increase in lead inquiry rate The service tone achieved a statistically significant increase in inquiries

Email Inquiry Rate Rel. Diff.

Control 1.58% --

Treatment 7.08% 349.41%


Background: National B2C, B2B financial company with an online and retail presence

Goal: To increase account activity, engagement

Primary research question: Which subject line will generate the most opens and clicks?

Approach: A/B/C/D split test (variable cluster)



# Subject Line TestedID1: [Name], Your Account Information is Ready To View

ID9: Sign on to your checking account

ID5: Your Account Information is Ready to View

ID3: Did you forget your [Bank Name] Online username or password?

ID10: Check your account balance online, anytime

ID6: Check your account balance online, anytime. Sign on now.

ID12: Don't forget to access your [Bank Name] account online

ID7: Sign on to your [Bank Name] checking account ID8: It's easy to log-in to your [Bank Name] account online

ID11: Easily manage your accounts in one place with [Bank Name] Online

ID13: Save time managing your money with [Bank Name] Online

ID4: Manage your money more effectively with [Bank Name] Online

ID2: Save Time by Paying Your Bills Online













Open30.77%30.70%28.49%28.21%26.97%25.23%25.21%24.43%21.26%20.44%19.32%19.27%16.53%














Open30.77%30.70%28.49%28.21%26.97%25.23%25.21%24.43%21.26%20.44%19.32%19.27%16.53%


# Subject Line Tested

ID1: [Name], Your Account Information is Ready To View ID9: Sign on to your checking account ID5: Your Account Information is Ready to View


ID10: Check your account balance online, anytime ID6: Check your account balance online, anytime. Sign on now.

ID12: Don't forget to access your [Bank Name] account online ID7: Sign on to your [Bank Name] checking account

ID8: It's easy to log-in to your [Bank Name] account online ID11: Easily manage your accounts in one place with [Bank Name] Online

ID13: Save time managing your money with [Bank Name] Online ID4: Manage your money more effectively with [Bank Name] Online



Open30.77%30.70%28.49%28.21%26.97%25.23%25.21%24.43%21.26%20.44%19.32%19.27%16.53%

Click4.59%4.44%4.40%4.34%3.84%3.64%3.59%3.47%2.90%1.94%1.70%1.54%1.05%

Live OptimizationEmail Messaging Effectiveness

Partner ID: The New York TimesLocation: MarketingExperiments Research LibraryTest Protocol Number: TP2802�

Background: Internationally recognized news service known for their journalism.

Our Goal: To significantly increase subscriptions.

Primary research question: Which email will generate the greatest subscription rate?

Approach: A/B split test (variable cluster)

Live Optimization eme = rv(of+i) – (f+a)

The Subscription Product Offering

A subscription to the NY Times• Home delivery of the Print edition• Unlimited, unrestricted access to articles

online, in their smartphone and tablet apps• Free digital access for one family member or

friend• Subscriber only special offers, discounts,

invitations

Incentive

• 50% off for the first 12 weeks

Live Optimization eme = rv(of+i) – (f+a)

The List

THE CONTROLSubject Line:

Open this now for Special Savings

BEFORE AFTER

62% Increase in Conversions The treatment increased the subscription conversion rate by 62.1%

Email +Subject Line

Open Rate* % Rel. Change Click

Rate* % Rel. Change Conv. Rate* % Rel. Change

Control 13.50% -- 0.67% -- .008% --

Treatment 15.60% 15.5% 1.01% 50.7% .013% 62.1%99% 96%99%

*Note: Absolute data points are anonymized, relative differences are actual

ReviewEmail Messaging Effectiveness

Perceived Cost

rvof

ifa

Perceived Value

eme = rv(of + i) – (f + a) ©

Pay attention to the change in the mind; not the change in the content

Review

• There’s a story behind everything that’s bought. This occurs on two levels: internal and external

• It is important to distinguish the difference between the product offer and the click-through offer.

• Consider the use of artificial urgency as an incentive to take action

• Aggravation in email messaging can occur on two levels: length and difficulty

• The natural level of concern in email is strong; do not try to mitigate it rationally.

Review

F Key Principles


DAN ARIELY NOAH GOLDSTEIN

MECLABS.com/Email

Plus, brand side marketers and tactical training sessions

The longest running research-based email marketing

event

February 17-20Aria Las Vegas

FLINT MCGLAUGHLIN