1

Peter Ciepiela

Evolution of an Optimization Team

Digital Optimization Lead

2

Optimize the Digital user experience of Philips.com…

65 countries 35 languages

300 Million visitors per year B2B & B2C

3

WHO AM I?

4

5

We faced some trouble…

Long time between A/B tests- 1 test in first month- 2.5 tests / month

Time to release results slow- ~4 months!

6

Weekly rhythm

…and needed to become more AGILE

Poker / Prioritize ideas

Pre-measure

E2E POOMA meetings

7

Number of tests / month:

Average number weeks to deployment:

Before After

7

21

6.8

3

*First 6 months starting February 2016 vs last 6 months

*

8

…more and more advanced tests via LEAN prototypes

9

10

Both tests failed…

11

…on an island

Org little understanding of the approach

No consistent way of sharing and remembering insights

Not informed from marketing context

12

13

Engage: Predict which test wins and win prizes

&

The Improvenator

Retain learnings: Repository for data, and insights from tests

Educate:Overview of UX, marketing, and Psychological principles

14

15

V1CONTROL

V2ICONS W/CONTENT

V3 ICONS W/O CONTENT

16

V2ICONS W/CONTENT

Buy intention +33%

Insight:Making important user goals such as navigation clearly visible on a website increases ease of navigation and increases buy intention.

Cultural context

Markets engaged

Validated for West

17

Engaging to inform & teach

In the numbers

Jeroen Marttin, 2015 Digital Insights Champion

1,050 Marketers on the Distro

235 that participated in 2015 (22%!)

40% get the right answer / week

18

is great… but…

Statistical validation

Fast incremental changes

KPI-driven validation

Won’t change experience

Validates only a snapshot

A B/T E S T I N G

Nike vs. Philips User test Benchmarking

19

Usability Benchmarking: Ease of finding a product (start Corp Home Page)

Adding more qualitative RESEARCH…

20

…surveys quickly learn about consumers

21

a prototype is great… but…

Paints a clear vision of experience

Shows concrete solutions

User-driven validation

Not data-verified

If KPI’s go up, don’t know why

Risk KPI’s crashing

Long time to deploy

Meanwhile…

22

Top navigation

- My Philips login (including text)

- Shopping cart

Striking search bar (118)

- B2C Navigation (359)

- Hero products or sub category images.

- No carousel

Promotion banner

Support widget (out of scope)

Predictive / adaptive search

No social icons

No contact slider

Navigation in the footer

- Promotions color (106)

23

Size of the main image.

Hamburger vs menu(#72) Striking search bar

No news letter widget available

Philips tweets removed

Long list of newest products

removed

Footer navigation

24

We evolved these into one Lean UX approach across teams…

25

IDEAL STATE TEST & VALIDATE LAUNCH PIXEL-PERFECTINITIAL STATE

Example: Lean UX evolution of Philips Home Page

26

By teaming up into one way of working across teams, we’ve been able to drive up conversion rates +67% in two years!

First A/B Test

78 A/B tests done1st Shop Test

Campaign Testing

Lean UX

166 A/B Tests

Targeting Test

New Nav Test

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

3.5%

Buy Lead Conversion Rate

27

THANK YOUlinkedin.com/in/peterciepiela

28