The Heap Guide to Activation · “AARRR” acronym. In this eBook, we refer to these metrics as “growth metrics.” Each of these metrics represent an important part of your customer’s

A Smarter Approach to Product Analytics

THE HEAP GUIDE TO ACTIVATION

The Heap Guide to Activation

Free trial

1The Heap Guide to Activation • heap.io

Introduction

Section 1: Why Activation Matters

Unlock Your Product’s Value Proposition

The Role Activation Plays in Growth Metrics

The Difference Between Activation and Engagement

Section 2: How To Measure and Improve Activation

The Right Way to Measure Activation

The Role of Behavioral Data in Activation

A Five-Step Framework for Measuring and Improving Activation

Using Analytics to Improve Activation

2

4

5

6

8

10

11

13

14

18

20

21

23

26

28

Section 3: How Heap Measures Activation

What Activation Looks Like at Heap

Finding Our Activation Metric

How Heap Uses Onboarding to Improve Activation

Closing

Table of Contents

The Heap Guide to Activation • heap.io 2

Introduction

INTRO

Activation

III

V

II

I

IV

HYPOTHESIS

HYPOTHESIS

HYPOTHESIS

HYPOTHESIS

HYPOTHESIS


Introduction

Getting new customers in the door is one feat for product folks. Getting those same customers to see the value in your product (and become lifetime users!) is an entirely different one.

Welcome to the Heap Guide to Activation. Our goal in writing it is to give product teams more opportunities to provide undeniable value for their customers. In particular, we believe that product teams can increase activation most effectively when they take a data-driven, experimental, iterative approach to product development, one that involves forming and testing hypotheses, figuring out what to measure, making small improvements, and learning from every single experiment, successful or not.

At Heap, we define activation as the very first time your customer realizes the value your product provides. In what follows, we describe why activation is a critical metric for product teams and what hypothesis-driven product teams can do to measure and increase activation.

We’ll also provide examples from our own practice that suggest how product teams might go about developing and testing hypotheses that address activation in their own products. At the core is the idea that a product manager’s job, when done right, involves using analytics to focus on problem areas and then creatively designing ways to address them.

As you read, you’ll realize that many of our strategies are about pursuing smaller wins. At Heap, we believe that incremental improvements, when implemented constantly and repeatedly, pay enormous dividends. As long as you’re being creative with your ideas and both rigorous and methodical about testing them (including recognizing when hypotheses are wrong and having the humility and ambition to change directions as needed), you’re doing it right.

Read on to improve your product or feature!


Why Activation Matters

01 S E C T I O N O N E

Activation

Acquisition

Retention

Revenue

Referral


Unlock Your Product’s Value Proposition

Think about a time when you met someone new. Maybe it was the confidence in their handshake. Maybe it was their warm smile, or the friendly back-and-forth banter. Either way, something in that initial interaction made you feel an immediate connection.

When you think about the ways that customers engage with your product, activation is that first impression. It’s the moment when new users form their first impression of your product, and then decide if they want to get to know it better. When activation is done well, it helps users unlock your product’s value proposition by getting them invested early on.

Activation also helps minimize user churn. Not surprisingly, users who find value in a product tend to stick around longer than those who don’t. By getting users to experience the “a-ha!” moment in your product — that critical moment when customers experience the value you’ve promised them — you’ll not only improve net retention, but you’ll also improve the growth of your business.

Being able to identify and understand that something that motivates new users to progress through the next stages of the customer life cycle is precisely why activation matters: it helps you understand what makes new users feel connected to your product and what makes them become loyal customers.

In the sections that follow, we’ll take a closer look at the role that activation plays in your growth metrics, outline methods for defining, measuring, and improving it, and explore some of our own learnings about activation at Heap.

“When activation is done well, it helps users unlock the value proposition of your product and gets them invested early on.”

“


The Role That Activation Plays in Growth Metrics

Product managers have multiple metrics to analyze across the customer lifecycle. We’ve outlined some common ones below.

You might know these metrics as “pirate metrics” because of the “AARRR” acronym. In this eBook, we refer to these metrics as “growth metrics.” Each of these metrics represent an important part of your customer’s life cycle. You also need to understand each of these in order to have a successful business.

Most companies can’t improve all of these metrics at the same time. If you’re focusing on all of them at once, it’s difficult to understand which experiments, efforts, and changes drive the most impact.

How do you know where to prioritize your efforts? At Heap, we often tell our customers to focus on one area of the customer journey at a time. This doesn’t mean that you discard efforts around the rest of the metrics, but it does mean that you align your organization around a single goal, and then prioritize that goal for a period of three months, six months, or even a year.

AcquisitionCustomerssigning up

ActivationFinding value

in your product

RetentionContinue to come

back over time

RevenueRenewals, up-sells,

cross-sells

ReferralCustomer advocates

on your behalf


The metric that we recommend our customers focus on most often is activation.

Why? Three reasons:

01 Activation is often the metric that product teams focus on the least.

Many people focus on getting new users in the door (acquisition) and then retaining them (retention), but they ignore the critical step in between.

02 Activation is one of the easiest metrics to improve.

Your users already have one foot in the door — you just need to entice them to take the next steps by activating them! Activation is the bridge between gaining new customers and turning them into repeat customers.

03 If you can get more users active in your product, other metrics tend to follow.

Users who are activated will stick around longer, they’ll renew their contracts and make more purchases, and they’ll even recommend you to their friends, colleagues, and strangers on Twitter.


The Difference Between Activation and Engagement

We commonly find that people confuse the term “activation” with “engagement.”

To highlight their differences, let’s talk about what activation actually means. To quote Growth Consultant Lincoln Murphy at Sixteen Ventures:

Why is Murphy’s quote so useful? Because it describes activation as a single moment. Unlike engagement, which measures the frequency and ways in which a user engages with your product over time, activation designates a single event or small series of events.

Another way to think about activation is as the “a-ha!” moment in your product. It’s when your customer first realizes the value that your product provides. Generally, when we think about activation, we’re talking about an activity that happens early on in the customer’s experience with your product or site.

Activation is the moment your customer derives value from your product.”

“


The Rundown: Quick Differences Between Activation and Engagement

Activation

Activation motivates customers to start using your product. It produces repeat visits and conversion.

Examples of activation include:

— Visiting a certain set of product pages or performing a specific number of clicks.

— Using the basic features and functionality of a mobile app.

— Starting the shopping process by completing a transaction.

Engagement

Engagement focuses on nurturing repeat and active visitors who frequently use your product.

Examples of engagement include:

— The feeling of being involved in a particular activity.

— The period of time during which a user stays involved with your product.

— Customers who come back and interact with your site multiple times.


How to Measure and Improve Activation

02 S E C T I O N T W O

Conversion

Blog Visits


The Right Way to Measure Activation

Before you can improve activation, you have to first understand how you should be measuring it.

We’ve found that many product teams start by measuring activation one way, only to learn later on that the measurement they’ve been using doesn’t quite do what they want it to.

For example, many product teams measure activation by looking at a report or dashboard similar to the graph below.

In this case, we’re looking at a simple graph of users who had a visit (also referred to as a session) within the past month. (These users are also known as monthly active users, or “MAU.”) Product teams might also look at daily or weekly visits to measure these users as well.

A recent Heap poll found that over 80% of respondents measure activation this way. However, the problem with this metric is that visits don’t actually signify that your customer is getting value out of your product.

80%Of respondents said they measure activation with MAUs, according to a recent poll.

Size of Monthly Users


F O R E X A M P L E

Think about the following scenario for a gaming app. In an effort to drive more daily visits, your gaming app institutes a daily rewards system. Because of this, users start coming in every day to redeem their treasure chest, coins, or extra points — and cause daily activation charts like the one shown to spike.

At first, everyone on your team is happy and high-fiving. But there’s a problem: these users are coming just to collect their prize. They don’t even play the game. What’s worse is that after a few weeks, they stop coming back.

So, the spikes that you saw? They didn’t actually do anything for the business in the end. This is just one example, but it does highlight the problem of equating visits with value.

A N O T H E R E X A M P L E

Consider a SaaS tool whose users login everyday because they have to, but they hate the experience. This might be okay for a while, but as that contract comes up for renewal, you could be in for a surprise when the account decides to churn.

If there are flaws in measuring activation this way, why do so many people do it? To a certain extent, because this is how it’s always been done — this is what legacy tools have taught us to do. When all you have is page view and session data, it makes sense that we would look at these and assume they captured value.

In the next section, we’ll explore a better approach to measuring activation.


they’ll do in your product. In digital products, however — especially B2B SaaS products — demographic data usually ends up telling you scant little about activation, conversion, retention, or feature interaction. This is because demographic data is broad, not granular, and by nature applies heuristics across a wide group of people.

Analytics tools have advanced in recent years, and there are now tools that allow product teams to look at behavioral metrics. Behavioral metrics are more about the specific actions people take in your product. Behavioral segmentation turns the demographic model on its head. When you take a behavioral approach, you track what people actually do in your product.

You see what behaviors tend to correlate with other behaviors. You can segment user groups with whatever degree of granularity is most effective. You can see what sorts of activities predict future activities. And so on.

Once you see how people behave in your product, you can then see how much any given behavior correlates with the metric you’re trying to track (in this case: activation).

Let’s examine how you can use behavioral data to better measure activation.

The Role of Behavioral Data in Activation

For decades, companies have segmented their customers using demographic data. Though details change, these tend to follow a familiar formula. Men over 5’10” who wear Asics tennis shoes and drink domestic beer tend to vote for a certain political candidate. Single women between 30-35 who went to Northeastern colleges and currently have no children prefer their kombucha in a clear bottle. Engineers at companies with 3000+ people prefer a specific SaaS product.

Certainly, knowing who is likely to buy a certain flavor of mouthwash can help marketing efforts. But in most digital products these demographic data points tend to be less useful for predicting if that group will get value from your product.

The general theory behind demographic segmentation is that knowing which group a user comes from can help predict what


A Five-Step Framework for Measuring Activation

How do you start measuring activation beyond just active visits to your site? Equally important, how can you discover what your activation metric truly should be? At Heap, we recommend using this five-step framework.

STEP 01Start trackinguser behavior

STEP 03Use data to validate

your activation metric

STEP 05Repeat!

STEP 02Define

“active”

STEP 04Make hypotheses

and run experimentsto test them


02 Define “active”

Once you have this data, the next step is to come up with hypotheses about what your activation metric is. Get a cross-functional team together that includes members from your sales team, customer success team, marketing team, and anyone with a good understanding of your audience and what they want to get from your product.

As a team, throw out ideas on what that “a-ha!” moment might be for your end users. In general, an “a-ha” moment will be the moment your customer realizes that he or she is getting value.

01 Start tracking user behavior

The first step in this process is to simply start tracking user behavior. If you are a Heap user, you’re already ahead of the game because you’ve already collected this data with Heap’s auto-capture feature.

If you aren’t tracking user behavior, start as soon as possible. Why? Because your customers’ behavioral data includes the biggest leading indicators of how your users find value in your product.

F O R E X A M P L E

For example, when we look at some well-known activation metrics, such as adding seven friends in 10 days or following 30 users in your first week, they make sense: You’re not going to want to log in to Facebook every day if there isn’t activity for you to view and people to interact with.

Twitter wouldn’t be very interesting if there weren’t a stream of topics or thoughts that you were interested in.

Another way to think about it: If Facebook measured activation solely on the frequency of user logins, it wouldn’t represent the ongoing value their users are getting.

Sure, a new user could log in daily for a few weeks, but if they don’t take any action to connect with the larger community, they will quickly lose interest and stop returning to the site.

Think about the key moments that might happen early in the customer’s journey and come up with a list of at least five behaviors or actions that you think could qualify as an “a-ha!” moment.


03 Use data to validate your activation metric

The next step is analyzing the impact of your activation metrics. For example, what sort of impact should we analyze? To the business? On other metrics? Are you saying that you want to see how well that metric tracks other key metrics (like conversion and retention)? Or you want to see how well that tracks with revenue? Both?

Simply coming up with guesses isn’t enough for data-driven organizations. Product managers need to then turn them into hypotheses, analyze them, and look at them in a behavioral analytics tool.

04 Make hypotheses about what might improve activation, and then run experiments to test them

Take action! At Heap, we believe the best way to increase activation is to come up with lots of hypotheses, then run experiments and measure the results.

Lots of product managers tend to make gut decisions about what to change, and don’t hold themselves accountable if their hypotheses turn out wrong. We think it’s great if things turn out wrong because it helps you understand what didn’t work — and why!

So, once you’ve identified what this activation metric is for your organization, the experiments can really begin. Now you’ll want to actually run experiments aimed at getting more active users.

Come up with experiments and strategies that you can try to get more users to perform these behaviors, sooner. Sometimes customers move through the growth stages naturally. More often they don’t. That’s ok — those moments of failure are opportunities for you to intervene! How can you re-engage past users who were never truly activated? Is there an opportunity to nudge your customers along with a new user experience, in-app guides, or a new user email campaign?

F O R E X A M P L E

Do users that perform those actions actually retain at a greater rate? Look at the behaviors that retained users perform. You also want to look at whether or not your churned users also performed that action. Once you find the metric with the greatest variance, you can define that as your activation metric.

17The Heap Guide to Activation • heap.io 17

05 Repeat!

The last step in this cycle is to simply revisit it periodically. As your business and customer base grows, you’ll want to re-evaluate your activation metric over time to make sure that you’re driving the right behaviors for the right group of users.

F O R E X A M P L E

An early startup might find that an activity such as scheduling a report is an indication of activation. As the company grows and its product evolves, it may later find that usage of a different feature set is a better activation indicator.

18The Heap Guide to Activation • heap.ioThe Heap Guide to Activation • heap.io

01 Improve activation by optimizing your funnel.

There’s more to improving activation than just picking your activation metric. One of the benefits of a tool like Heap is that it lets you examine all sorts of things about that metric.

F O R E X A M P L E

What do people do right before activating? Right after? How smooth is the path to activation? How easily discoverable is the action you use to measure activation? Using a product analytics tool like Heap gives you deeper insight into your funnel analysis so you can pinpoint the areas that lead to greater activation.

Say you realized that right before activating, people tend to visit your welcome page to learn about and explore all the different actions they can do in your product. This could lead you to test all sorts of experiments to drive activation, such promoting the welcome page more prominently across different areas of your site, marketing campaigns, or throughout different onboarding activities.

Using Analytics to Improve Activation

For product teams focused on finding ways to improve their activation rate, diving into a product analytics tool is a great place to start! A tool like Heap can help product teams better understand the path to activation, how easy it is to discover your product’s value, and the different ways marketing efforts might help (or hurt!) activation rates.

Here are three areas where you can use product analytics to improve activation.


02 Improve activation with behavioral segmentation.

Then begin to experiment with behavioral segmentation to understand your users better. A product analytics tool can quickly help you answer really specific questions about where your users are coming from and how easy it is for them to findvalue in your product.

03 Improve activation through more focused marketing strategies

Lastly, don’t forget about the role of marketing in activation. Product analytics tools are equally insightful for your marketing team and can help them identify ways to improve campaign performance.

Asking and answering these types of questions is how people use analytics to improve their growth metrics.

F O R E X A M P L E

Which channels do people come from, and which of those are most likely to increase activation? Which demographic groups are more or less likely to activate (job titles, company size, geography, etc.)? How easy is it for people to get to those “a-ha” moments? What other actions in the product are people likely to take, and how do those correlate with activation?

Another way to look at it: Think about how different people buy tennis shoes. Some people look for comfort, others base their decision on a specific color or certain brand, while others are influenced by cost. wYour users are motivated in a similar way. You might have two users who share similar demographic profiles, but have very different ideas of which features are the most important to them. Understanding how your users behave and which features matter to them will help you better understand how to activate them.

F O R E X A M P L E

Consider your strategic marketing efforts and how those affect activation. Do people who receive emails tend to activate at a greater rate? Do they use coupons? Were they in your nurture cadence? (And so on.)

Take a closer look at your marketing touch points leading up to activation to learn whether a specific type of offer, such as a how-to guide or a special discount, motivated users to activate in your product. Another opportunity? Combine your insights from behavioral segmentation with marketing outreach to deliver more targeted messaging that can help activate high-value customers.


How Heap Measures Activation

03 S E C T I O N T H R E E

AR

DAY 0

10

20

30 60 90

18

2

12

6


What Activation Looks Like at Heap

Data drives our own business decisions at Heap, notably within our product and customer success organizations. Heap’s net retention is one of our top company metrics, and many of our business decisions (including how we approach activation!) are focused on improving it.

Improving our net retention depends directly on our ability to drive customer value in the onboarding phase. Activation is centered on getting our customers onboarded successfully with a high-touch onboarding process that includes key customer-driven success milestones, such as running queries, saving reports, and using our group-by feature.

What does activation look like at Heap? We used our five-step process to uncover the actions that help our customers get to their “a-ha!” moment quickly.


Similar to the MAUexample in the previous chapter, we also used a simple definition: a user having a session in the past 3 days.

Early on in our company journey, we used a single metric to define whatactivation meant to us.

For example, customers that were returninghigher than one sessiona day in the past 30 days didn’t indicate that they were going to renewtheir subscription.

We knew that activation is much more complex than having a session.

We evaluated different actions – from events query in the UI and syncs to a BI warehouse – to learn how to help users get to insight quickly.

We used our five-step process to what actions we wanted customers to take during onboarding and what value to deliver to them.

Our customers needed help to move away from basic metrics, such as count of sessions or count of page views,to ask deeper-levelquestions by runningmore advanced queries.

When we looked intothe data, we identifieda couple differentsignals that would result in people becoming active users.

We uncovered that running a query and saving a report were two actions that led usersto insights as quicklyas possible.

Additionally, both actions helped produced high user retention rates when lo�ing into Heap.

Heap’s Journey to Finding Our Activation Metric


Finding Our Activation Metric

At Heap, we found two metrics that correlated strongly with activation: “Saving Reports” and “Group By.”

Saving Reports

We think of our value as delivering actionable insights to our customers. There’s no easy way to measure that we’ve delivered this, but we ideated that if you save a report, it’s a good indicator that you found a data point or insight meaningful enough to revisit it later.

After spending time with our metrics, we realized that when someone saves a report in their first session, they are 2x more likely to be long-term users of Heap. No other activation event/definition had this same effect.

With this information in hand, we looked more closely at how users execute queries and which queries were then saved as reports. We needed to better understand the relationship between executing a query and becoming a long-term user. Was this a correlation or a causal relation between the two?

Through our experiments, we found that we had an opportunity to not only help our users execute more queries, but also help make the query meaningful enough to reference it later on and save it as a report.

This was an indication that if we can uplevel the questions our users ask in Heap, they will have a higher retention rate, and therefore a higher activation rate in Heap.

Retention

Group By

Filters

Compare Segments

Date Range

First Time

Start Event

Start Event

View - Any App Page

Return Event

Add Filter

Execute Query

Dec 31, 2016 - Apr 27, 2018 by Week

Add Segment

Has done sequence Accept Invite → Save Report

Date First Seen after 01/01/2017

Retention of users who did View - Any App Page and then Execute Query, whereDate First Seen after 1/1/2017, by Whether User Has Done Sequence, then undefined


Group By

The same logic applied to our second metric, “Group by,” a query functionality that helps users get a deeper understanding of something, such as count of sessions, that is grouped by device type.

This metric told us that when a Heap user uses the “Group by” feature when running a query, their engagement rates tend to be 2x those of users who don’t use Group by.

In addition to running a query, we found that both saving a report and doing a Group by were more focused definitions of active users in Heap. With this information, we could then analyze different ways we could help deepen the questions that our customers ask in the tool and ultimately drive higher activation rates. Has Run a Query

w/ GroupBy

Has Run a Query

51.86%

21.85%

(-10.15%)

(-22.84%)

After 10 months

Did Su�ested Reports improve query retention?

Retention

Group By

Filters

Compare Users

Date Range

First Time

Start Event

Start Event

View - Any App Page

Return Event

Add Filter

View - Any App Page

Past Year by Month

Add Group By

vs.

Has Run a Query

Has Run a Query


Using Data to Address Customer NeedsBy uncovering this information, our customer success team could address these potential issues early on with our customers, and provide them with the resources, tools, and support they needed to not feel stuck.

Along with the data we pulled from Heap, we also looked at qualitative data from our customer base, including their reasons for churning, to learn about some of the issues that customers were having. This gave us key insight into areas we could focus attention on during the onboarding process.

For example, some of the common questions customers and churned customers asked were:

— How can Heap be less of a blank slate?

— What questions can I ask, and how do I ask them in Heap?

— How can I share analyses in a dashboard with my coworkers?

— How can I easily define events in Heap?

F O R E X A M P L E

To help customers understand the kinds of questions they can ask in Heap, we updated our in-app walk-through to guide users through the process of running a query and saving a report. We also made sure our onboarding sessions covered this material and even designed a new feature called “Suggested Reports” to help address some of those pain points.


How Heap Uses Onboarding to Improve Activation

The onboarding process plays a significant role in improving your activation rates. Onboarding is the process by which new customers become familiar with your product’s features, functionality, and overall business value.

When it’s done well, a successful onboarding process helps new users uncover the value proposition of your product more quickly.

It’s for these reasons that we looked to our own onboarding process at Heap to help our customer services team activate customers.

We created a product-health metric to help our customer success team further gauge whether or not a customer was truly activated and ready to leave the onboarding phase.

In addition to the activation metric we outlined in the previous section, we wanted to keep a pulse on how active our users are in the product during the onboarding phase. This gives our customer success team an idea of which customers might need more support using the tool and running queries or saving reports.

We combined our Heap data from the UI with usage data from our product warehouse to create health metrics.

In doing so, we decided to evaluate customer health using color-coded activation metrics.

If the account is under 30 days old, the customer health level is green.

If the account is 30-60 days old, the customer health is yellow if there are fewer than 10 active events. Otherwise, it’s green.

If the account is greater than 60 days old, customer health is red if there are fewer than 10 active events. Otherwise, it’s green.

GREEN

YELLOW

RED


The indicators help signal to our customer success team how to adjust their onboarding strategy to help customers query more events. They also serve as a red flag for customers who are not adopting the product and enable our customer success managers to take corrective action to improve customers’ health score.

We also use these metrics to quickly identify at-risk customers early on in their life cycle rather than waiting until renewal to discover that they never saw value or are under utilizing the tool. Account managers who have yellow or red customers in the early stages of their life cycle might offer additional training, point their customers to helpful webinars, or identify new champions within the account.

They might also spend more time reaching out to uncover what the blockers are. Customers who are green are often sent more content about upleveling their usage through more advanced features or use cases.

Customer Journey

1 2 3 4 5


Closing

When it comes to how you measure a customer’s interaction with your product, activation is the one that can help you understand what — exactly — makes new users feel connected to your product so that they become loyal customers.

In this short book, we’ve tried to outline the role that activation plays in growth metrics, offer methods to help you define, measure, and improve activation, and share some of the lessons we’ve learned at Heap during our own journey.

At Heap, we believe that the key to improving activation is to develop a hypothesis-driven, experiential, and iterative approach to product development. Our product framework is focused on helping product managers probe for opportunities to develop their product, then to capitalize on those opportunities, knowing that an ongoing series of small wins can add up to or pave the path toward major innovation.

We hope this has been helpful to you. If you have questions or thoughts about any of the material in this eBook, or simply want to know more about Heap’s approach to product management can be useful for you, we encourage you to get in touch at [email protected].

CLOSING

mailto:support%40heap.io?subject=

www.heap.io

Documents

The Heap Guide to Activation · “AARRR” acronym. In this eBook, we refer to these metrics as “growth metrics.” Each of these metrics represent an important part of your customer’s