PowerPoint Presentation
Building Data Driven Customer Experiences
Ice breaker: Quick show of hands, who here has ever bought a
pair shoes online? Keep your hands up if youve had to return a pair
because either you didnt like them or they didnt fit when you
received them. Now, how many of you enjoyed the process of
returning that purchase?Introduce myself:Haley Nemann Director of
digital experience at crocsCrocs is a casual footwear company based
in Boulder, Colorado 11 sites globally The main focus of my role is
Global Ecommerce Customer & User Experience & Optimization.
Youll notice that I jump between the terms user experience and
customer experience. These two are not one in the same, but in the
world of ecommerce theyre pretty close. Our customers are also our
users. So by improving user experience we are essentially improving
our customer experience at the same time.A main focus of my job is
prioritizing and approving all front feature improvements we make
to our global platform and online experience.To aid in this process
we do a lot testing at crocs everything from user testing, to MVT,
to testing configurations within our platform. Today I am going to
talk about how to use data to influence changes to your CX that may
seem unfavorable to your businesses bottom line.
1
Good UX is {usually} great for businessHistorically customer
experience has not been tied to business outcomes
Universal need for data driven experiences, or measurable user
experience
The Quantitative vs. Qualitative Debate
BULLET 1)Historically customer experience has not been tied to
business outcomes however I seek to change that!A large part of my
role at crocs is to focus on using data to drive insights and power
better experiences for our consumers. Happy customers come back!
Unhappy customers often spread the word of social platforms.To be
successful, I must first understand user behavior patterns and
align our site experience accordingly.BULLET 2) There is a
universal need for data driven experiences, or what I call
measurable ux. This is challenging for todays business leaders
across most verticals and industries, not just retail.BULLET 3)The
Quantitative vs Qualitative Debate This is an ongoing debate within
our ecomm department and Im sure many of you experience this debate
as wellAccessing the data itself can be challenging, and that data
can often be inaccurate, laden with biases, flaws and outdated
information.In my case, data isnt just about numbers Its a blend of
quantitative and qualitative information from the field, our sites,
testing and customer feedback. Looking at the data from a variety
of vantage points helps minimize biases. In a few minutes we will
go into a real world example of how we combine these two types of
data to inform business decisions.2
Define: Analyze business goals & translate them into UX KPIs
Benchmark: Measure the current UX to establish a baselineForecast:
Predict the likely impact of improvement Design & Build: Design
& develop guided by UX KPIsMeasure: Test & collect results
data to measure successAnalyze: Use results to provide actionable
insights, make recommendations and quantify a value of success
Ensure experiences are intentional and tailored to deliver
business results
Methodology
SLIDE 3Bullet 1) There is no exact science to creating data
driven user experience, but heres one approach to help ensure the
experiences are intentional and tailored to deliver business
results:Subsequent bullets) Define: Analyze the business objectives
and translate them into UX KPIs and metrics- Example of this, we
think our checkout process is too long. Our goal is to reduce fall
out and have users complete this process in X number of seconds
without making errors in the processBenchmark: Measure the current
experience to establish baseline- Following along with our checkout
example, This would entail user testing the current experience,
watching videos to identify friction points and understanding how
long it is currently taking the user to complete the checkout
process. This would also involve looking at your checkout funnel
analytics and identifying drop off rates for each step of the
process.Forecast: Predict the likely impact of improvement At crocs
we call this PIE, an idea we adapted from the website the wider
funnel. This is calculation of Potential, Impact and Effort that
helps us measure the affect of a particular feature. Our analytics
team came up with a fantastically nerdy calculation that looks a
combination of traffic, what section in the shopping journey we are
attempting to impact, plus expected improvement to help with this
PIE calculationAgain in our example, you may expect checkout to
have a high potential for impact because users are so far down the
funnel, however the effort and risk are very high. Therefore this
will likely extend your effort timeline to account for additional
risk mitigation strategies like extensive QA & user
testing.Design & Begin: design and development the new
experience guided by the UX KPIs- At this point you may build the
entire feature, or just a prototype to put in front of testers and
iterate uponMeasure: Test and collect results data to measure
success- The may be in the form of an A/B test prior to launching a
feature at 100% or as post launch analysisAnalyze: Use results to
provide actionable insights, make recommendations and quantify a
value of success
3
Real world example
REAL WORLD Example:We know offering free returns to our
consumers is the correct CX, however we were unsure of the impact
to our bottom line. We needed to determine if offering free returns
would increase our CR and RPV enough to off-set the cost of the
return shipping and touch cost of re-stalking the product. In the
past we have done extensive testing on our free shipping thresholds
to determine that optimal price. However now we needed to
understand the impact of free returns.We ran an extensive test set
out to answer these questions.
Question: To offer or not to offer?
Test Plan:Competitive researchWhat are customer expectations?
Test A/B test of the free returns offering Measure Qualitative:
Industry research, user testingQuantitative: Conversion Rate,
Revenue per Visit and Returns Rate
Free Shipping, Free Returns
We combined both qualitative and quantitative results data to
answer our questions. For the qualitative we did competitive
research and user testingFor the quantitative - we ran an A/B test
of the offer by presenting different messaging to a test group live
customer on the front end of our the site. This was a bit tricky
because we dont have a order management system sophisticated enough
to know which messaging the customer saw on our website when she
made her purchase (free shipping, free returns) or (just free
shipping). To solve this problem fairly easily we actually decided
not to charge any customers for returns of purchases that were made
during the test period. So those customers who thought they would
be charged for return shipping simply got delighted when they
received a full refund amount. To Measure Results, we summarized
our research results and look at the CR, RPV and Return Rate.5
be generous with your return policiesboth your brand reputation
and your margins will benefit. IR, February 28, 201657% of shoppers
consider having to pay for return shipping as an issue when making
returns UPS Pulse of the Online Shopper, June 2015If customers are
forced to pay for returns, they will either: keep a product they
dont want, or reluctantly shell out to get one they do want. Either
way this will likely reduce customer satisfaction and ultimately,
customer loyalty. IR, December, 2015Customers who paid for their
own return decreased their post-return spending at that retailer
75%-100% by the end of two years. In contrast, returns that were
free to the consumer resulted in post-return customer spending that
was 158%-457% of pre-return spending. Journal of Marketing,
September 2012
Industry Research around the topic heavily supports the cause
for offering free returnsWe also did a competitive analysis of
other branded footwear manufacturers and found that 7/8 offered
free returns
6
Qualitative Results
As a part of understanding user expectations we ran a series of
user tests that included the very broad question what factors
influence your online purchase decision?. This question was
intentionally asked this way as to not sway the users answers.Im
going to show a series of user testing clips where users discuss
their general expectations when shopping for shoes online. The
video jumps around a bit as its edits of users just answering this
question.
Qualitative Results
As a part of understanding user expectations we ran a series of
user tests that included the very broad question what factors
influence your online purchase decision?. This question was
intentionally asked this way as to not sway the users answers.Im
going to show a series of user testing clips where users discuss
their general expectations when shopping for shoes online. The
video jumps around a bit as its edits of users just answering this
question.
Revenue per visit = +2.1%Conversion rate = +2.8%Customer
Experience =RPV
CR
Return Rate
CXRecommend Free Shipping and Returns
Results>Return Rate = +14%
Customer experience was measured qualitatively as a win. This
was based both on the finding from the user testing videos &
Journal of Marketing research mentioned earlier that found post
free return customers spending upwards of 158% of their of
pre-return spending. We know our consumers shop with us on average
of 1.5 times per year, so losing current customers could have a
vast impact on customer attrition. However in this particular case
the quantitative data alone was enough to quantify free returns as
a win. The increase in RPV and CR was enough to offset the increase
we had in returns because our returns baseline is low. This is just
small example of how you can use data to make a case to an
executive suite that the right Customer Experience can also be
great for business even though it may appear otherwise.
9
THANK YOU!
10
