Digital Optimization Benchmarking Study

Data-Driven Culture Testing Personalization

based on a survey of 150 ecommerce executives

Testing and personalization technologies have been around for a while. In fact, they’ve exploded over the last five years. Yet, it’s still early days for executing and integrating testing into decision-making and even earlier for personalization. Why?

Here’s the bottom line...

While the appetite for data-driven culture is present, implementation is lagging.

Why are companies stuck in the first inning of the data-driven game? To find out, we surveyed 144 eCommerce executives.

let’s get real for a moment.

This study is broken down into three sections:

Data Driven Culture

A/B Testing Maturity and Capabilities

Personalization Maturity and Capabilities

Followed by the data learning takeaway:7 Inside Secrets to Building a Data-Driven Culture

2

3

1

10 key findings

data-driven culture

Throughout the year, Clearhead hosts thought leadership dinners across the country. Each time we ask executives how would they grade themselves as a data-driven culture. Invariably, the answer always averages out to a C+. We wanted to learn more about this.

1

Only 1 in 5 respondents rated their company as having a ‘world-class’ data-driven culture.

According to our study, the average retailer would rate their data-driven culture maturity as 5 out of 10.

Further, and related, for those with testing and personalization capabilities, they rate themselves 5 out of 10 for USING testing and personalization to solve important problems that impact the business

Average Number of Full-Time Employees Dedicated to Optimization

What about staff to support data-driven practices? What about getting more actionable analytics? While there are, on average, four FTEs focused on web analytics, there are only two on personalization or A/B Testing. Consider this number with the fact that testing and personalization requires a more diverse set of talents, including product/project managers, data analysts, developers, and designers.

Data-Driven CapabilitiesThe low grades and short staff are in contrast with high aspiration. There is executive support to become data-driven, but gaps in process, people and skills are holding companies back from their data-driven goals.

testing maturity & capabilitiesAt the core of a data-driven culture is the use of data to increase decision confidence. A/B testing, then, becomes a core capability for making high-outcome decisions for UX, product design, technology and project prioritization.

2

Testing and Maturity CapabilitiesSo where do we stand on testing maturity and capabilities? According to our study, few retailers have “very present” capabilities for A/B testing:

Testing and Personalization Staffing Woes

Staff necessary to do testing and personalization are only somewhat available. 53% UX/UI, 68% of developers, 58% of analysts and 50% of the product managers required to do testing and personalization are either not at all or only somewhat available.

In addition to a lack of availability, the employees responsible for testing and personalization are only somewhat skilled.

How available are the people dedicated to testing and personalization roles?

How skilled are the people dedicated to testing and personalization roles?

*my company does not have people dedicated to this role

Testing TechnologyOn the plus side, the majority of respondents report having a testing platform in place, whether licensed or in-house build:

Those who do have testing in place rate their satisfaction with the tool a 5 out of 10:

Testing Velocity and ProgressFor those that have software, the vast majority run fewer than five tests a month:

Why is that? Our data implies that it is likely due to a lack of headcount and process rather than the technology itself.

Respondents rate their testing process as 5 out of 10:

Personalization and Maturity CapabilitiesWe find there is a lot of talk around “personalization”. And by talk, we mean brands express ambiguity, confusion and frustration with their effort to progress. That doesn’t mean personalization is unattainable. It means we’re early. And it suffers from the same operational gaps as testing.

3

Personalization Technology

However, for those that have technology they’d rate their personalization operations and process as 4.6 out of 10:

Similar to testing, we find that lack of technology isn’t the problem when it comes to personalization. While 64% of retailers have the technology, they tend to lack the process and rigor needed to execute their personalization efforts.

So, in the final analysis, if 64% of online retailers have a personalization technology on board, but approximately 80% have limited capabilities around data, people, skills or process to execute that technology, we are at an impasse. The full personalization ‘solution’ is not complete.

Seven Inside Secrets to Building a Data-Driven Culture

The majority of brands have software to test, personalize and use data, but have yet to bridge the gap between their technical/data capabilities and the people, processes and skills needed to achieve data-driven outcomes.

After working with over 60 clients on their digital journeys over the last four years, we’ve learned some tricks to accelerate a data-driven culture.

Here are seven we’d like to share with you

1: Product and UX Should “Own”

Experimentation

2: Invest in Experimentation Fluency

3: Practice is the Thing

4: Roadmap from Problems

5: Story Before Stats

6: Plan on Personalization Phase

7: The What and the “Why”

Interested in the full report? Read it now.

About Clearhead

Sam Decker, Matty Wishnow & Ryan Garner founded Clearhead. They’re a lot like you.

They founded Clearhead because they decided it was time to graduate from buzzwords and good intentions to real data-driven change & improvement.

As the world’s leading digital optimization company, Clearhead helps daring brands optimize their business outcomes through:

Continuous Optimization & Personalization | Data-Driven Redesigns | Data-Driven Culture Accelerators

Ready to accelerate your path to digital maturity? Let’s talk.

sales@clearhead.me | 844.425.7432 | clearhead.me

Interested in their stories? Read them here.

About the Participants

thanks for reading! clearhead.me