Getting started with server-side testing

Dennis Pavlina and Thomas Davis

WiderFunnel Strategist and WiderFunnel Web Developer

● The presentation deck and recording will be available to you after the webinar.

● Feel free to ask questions during the presentation, using the Question panel.

Housekeeping

Dennis Pavlina, WiderFunnel Strategist

● Oversees experimentation strategy for some of WiderFunnel’s most notable full stack clients

● Guest lectures at local design and tech institution, RED Academy

Your Presenters

Thomas Davis, WiderFunnel Developer

● The go-to guy for server-side experiment development

● Instructor and mentor at local coding bootcamp, Lighthouse Labs

● Why adopt server-side testing?

● Laying the foundation for server-side testing

● Technical best practices

● Fostering server-side testing long-term

What We’ll Cover

Why Server-Side?

What is Server-Side Testing?

What is Server-Side Testing?

The Possibilities of Server-Side

The Possibilities of Server-Side

The Possibilities of Server-Side

● Improved user experience

● Ability to open up testing to back-end changes

● Feature testing

● Faster implementation of winning variations

Server-Side Experiment Example: Algorithms

Displaying “Other Products

You Might Be Interested In”

as opposed to “Frequently

Purchased With” products

Server-Side Experiment Example: Onboarding flows

Server-Side Experiment Example: Trial durations

Identifying whether more

users sign up when given a

7 day trial versus 14 days or

30 days.

Evaluating Opportunities for Server-Side

Accessibility Depth Goals Channels

WYSIWYG vs. Dev team Styling vs. Functionality Engagement vs. LTV Web vs. Multi-Channel

Laying the Foundation for Server-Side Testing

Define your success metrics: Identify new goals

Server-Side

● Rate of visitors who complete all onboarding steps● Rate of visitors who convert to paid signup within 24 hours● Rate of visitors who follow 5 or more people during onboarding

Client-Side

● Signups● Blog subscriptions● Visits to key pages

Define your success metrics: Place value on feature testing

Define your success metrics: Set the right timeline expectations

Full Stack Best Practices

● Eliminate the difficulty of integration

● Principle of “separation of concerns”

● Easily testable

● Doesn’t intrude on feature builds

It’s all about Middleware

Activate the experiment

Monkey Patch the res.send method

Serve the correct variation

Original Route method remains untouched

● Moves targeting to the Optimizely tool / Optimization specialist

● Can target anything from pages to device types

● Experiments become more dynamic (Not hardcoded)

Targeting through Attributes

Find the current page, return as an object

Send attributes and return the correct variation

Working with SPA’s

● Use getVariation() to get users into variation without tracking them

● Route to activate user later / Front End JS SDK

https://www.myawesomesite.com/activate?test=wf_test&user=xxxxxxxxxxx

● Create top level A/B testing components if using React or Vue

● Setup all global events first (i.e. page visits, revenue tracking)

● May need a ‘AJAXable’ route for UI component goals /

Alternatively use the client-side JS SDK

● Make naming conventions specific and namespaced

products-index/add-to-cart-cta

Custom Event Set up

● Have a roadmap for the installation process

● Use cookies or webhooks when dealing with page caching reverse

proxies

● Random string vs. database user ID to identify the Optimizely user

● Test Driven Development (TDD)

Tips and “Got ya’s”

Get Optimizely X Full Stack

Get the Resources You Need

Gather and educate the right talent

Coordinate the effort via dedicated team leaders or an agency partner

Identify the Best First Experiments

Hypotheses Potential Importance Ease PIE Score

Hypothesis #1

Web analytics

Heuristic analysis

Voice of customer

Cost

Traffic volume

ROI

Technical

“Political”

8 7 7 7.3

Hypothesis 2Displaying a free trial countdown timer on the dashboard will increase the rate of users who convert to a paid account.

Identify the Best First Experiments

Hypothesis 1Removing all optional steps from the onboarding flow will increase the rate of users who convert to a paid account.

Hypothesis 3Automatically filling in location-specific information in the paid signup form fields will increase the rate of users who convert to a free account.

Identify the Best First Experiments

Hypotheses Potential Importance Ease PIE Score

Remove optional steps

Countdown timer

Pre-filled form fields

8 9 9 8.7

6 8 8 7.3

6 8 5 6.3

The Long-Term Vision

Combining Client-Side & Server-Side

Work to establish a “We should test that” culture:

● Dedicated brainstorming sessions● Regular lunch and learns● Review assumptions and status quo● Company-wide results/insights updates

Experimentation Culture

Q&A

Thank you!