Practical Price Testing

Jared Waxman

Pricing Challenge in Summary

Nichols & May “$65 funeral”

About Me

I have run over 500 marketing experiments

Many of the most successful tests have been related to pricing & packaging

I teach a/b testing courses around topics such as statistics and design of experiments

Have run experiments for...

Intro

Price testing isn’t just the price. It encompasses: Price levels Price gaps Price anchors Discount strategy Line up strategy Feature composition Cannibalization Pricing by Marketing Channel Pricing by Customer Segment Pricing by Vertical and more

Broader Impact

Pricing decisions don’t just impact immediate conversion. It also impacts...

customer satisfaction and loyalty ability to sell other things in the short-term refund rates future revenue stream possibilities brand perception customer expectations

Can you? Should you?

There’s some debate around the legitimacy of price testing against live customers.

It is done all the time.

Be careful not to discriminate. Randomization is not discrimination.

Be careful around consistency of experience Test targeting can be tricky in particular for

logged-out experiences Test quickly and have a back-pocket offer.

Toolset

Tougher to do with off-the-shelf testing tools

Often requires building own testing and reporting capabilities

Goals

As with any experiment, what are the key success metrics? Just because it’s a “price test” doesn’t mean everyone is necessarily on the same page at the start.

KPI options Top line revenue Price charged at checkout Bottom line profit short-term Bottom line profit long-term New customers NPS / WTR / WOM etc. etc.

Effort vs Reward

Price testing is at the harder end of the optimization spectrum

The payoff is well worth the investment however

I have seen > 40% lifts from price testing

Price testing has the broadest impact: not a page or segment usually but an overlay on top of the whole business – huge leverage

Lab vs Live

Ideal is to start testing in the lab (surveys, focus groups, conjoints, etc.)

Once you have pretty good confidence in certain directions, then move it to live tests

Does not have to be done in that order however.

Controlling Variables

Simplest approach is to test in phases: Packages (e.g., components of the offer) Lineup (e.g., how many packages there are) Messaging (e.g., how to sell each package) Prices for Core Offering (the actual price levels) Price levels for Add-On Offerings Timing or Sequence of Sales Cycle Discounts

Be sure that you are either testing just one of these factors at a time, or else establish an appropriate experimental design to tease out the effects of the different factors.

Continuous vs Discrete Variables

Price is continuous, can test more points on the curve without additional sampling overhead

Number of packages is not quite continuous 1 means no choice Odd numbers are best Limited to single digits

When to Price Test

You have new products

Your products have changed

Your competition has changed their pricing

You’re at a new part of the market adoption curve

You changed strategic directions

Subscriptions

SaaS? Have to use cohorts

Trial period? Have to track free to paid

Monthly subscription? Get a few points on the curve and estimate

Annual subscription? Harder to estimate

Choosing Your Customers

In some cases, the prices you choose determines the kinds of customers you’ll get.

The are downstream implications of acquiring different types of customers.

Lower list prices or frequently discounted prices bring in more more conscious customers. The may be: less loyal less likely to buy future offerings more likely to cancel more likely to use customer support

Discounting

You should study the frequency of discounts steepness of discounts duration of discounted period predictability of discounts impact on brand perception impact on loyalty impact on margins discounting wars with competitors

Scarcity

Scarcity is closely related to price

The price will go up soon

The discount will end soon

There are a limited number

This offer only available for customers from... / who...

Fake Door Testing

Possible in a modified manner with price tests, but can be tricky. Can pair with limited quantity messaging when done authentically.

More easily done with product

Free

A free version can bring in lots of potential customers

Offering a free version can be viewed as an advertising expense

Free can be important or even necessary from a competitive standpoint

Free can also impact brand perception and ability to sell very high end products

Free can also incur support costs

Free can be a way to get valuable customer feedback to improve the product

Free can be a way to build a network effect for certain offerings

Free has a cannibalization impact that can be measured

Free Trial

To test:

Credit card required vs not

Trial period

Auto-charged or not

Feature limits

What happens if customer doesn’t end up paying. Do any features remain? Any data saved?

Refer a Friend as Pricing

“Refer 3 friends to get an additional 1 GB of space” is a form of price... instead of paying you money, they are paying you in-kind with social-based marketing

How else can you “charge” without asking for dollars? content uploads P2P disk usage data or other ways to help build/grow the

product

Price Display Actual Price

Total amount Per Month, Per Day etc.

Perceived Price Valued at Compare at Price before of after discounts? Component Value

Price Display Strike though List

Discount Display % off $ off You save

Badges Best value Most popular

Magic numbers

Endings .99 .95 etc

Round numbers

Trick specificity

Package Hiding

Companies tend to ever expanding product flavors and package sets. Over time what started out as a rational for the business becomes a drag on conversion

Simplest way to approach price testing is to trim back the lines offered to a more management set (1 to 3)

Supplementary packages can be hidden completely or pushed down to deeper levels or segment-based marketing – excluded from general pop messaging

This approach will give some idea of loss due to hazards of excessive choice

Loyal Customer Pricing

Upgrader discounts

1st year premium

Minimum period

Quantity Pricing

Buy more save more

Usage based

Indecision Risk

Too many price value trade-offs and prospects bail out

Must also find right level of product detail to not overwhelm, but still sufficiently inform

Component Values

Hard to market all the features or benefits

Should quantify relative value of each and present as such

Each package must have one sufficiently high value component for important customer segments

Negative Example

Negative Example

Strike through only on one package

Huge jump between low and mid package

Highlight (“best value”) on most expensive package

Package/Pricing but no CTA based on that info

State fees buried elsewhere

Inconsistent messaging about differences between packages

Combination of transaction, subscription/trial, and lead gen not easily parsed

Positive Example

Positive Example

Odd number of packages

Center package is recommended

Pricing displayed at the lower-seeming “per month” amount

Original strikethrough price, % off highlighted & amount saved

Dead-simple benefits list; Clutter free design

Visual hierarchy of CTAs

Risk-free messaging

Minimal asterisking

This page was tested extensively:

Game Changing Examples

Amazon Prime

Amazon partner / affiliate store

Apple App store

Southwest Airlines

Netflix

Adobe Creative Cloud

Conclusion

Price testing encompasses a huge array of testing possibilities

Price testing requires a serious commitment and some finesse to get right

Price testing is likely the biggest lever you have