Is data science inhibiting product development?
Steven LemmChief Data Officer @ Outfittery GmbH
[email protected] | Data Natives 20162
WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)
[email protected] | Data Natives 20162
WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)
Example e-commerce platformFrontend products: registration/authentication, check-out, recommender engine, article search, …Backend products: inventory control, order engine, logistic processes, payment service, payment control, …
[email protected] | Data Natives 20162
WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)
Regis-tration
article search
article details
check-out
payment control
Example e-commerce platformFrontend products: registration/authentication, check-out, recommender engine, article search, …Backend products: inventory control, order engine, logistic processes, payment service, payment control, …
For the rest of the talk consider a simplified platform
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
PRODUCT DEVELOPMENT
3
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
PRODUCT DEVELOPMENT
3
Regis-tration
article search
article details
check-out
payment control
How to measure additional benefits?
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
KPI: search rank
KPI: revenue
PRODUCT DEVELOPMENT
3
Regis-tration
article search
article details
check-out
payment control
How to measure additional benefits?
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
KPI: revenue
article searchVer. B
KPI: search rank
PRODUCT DEVELOPMENT
3
How to measure additional benefits? A/B testing!
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
KPI: revenue
article searchVer. B
KPI: search rank
PRODUCT DEVELOPMENT
3
How to measure additional benefits? A/B testing!
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article search
article details
check-out
payment control
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score
payment control
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article search
article details
check-out
payment control
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article search
article details
check-out
payment control
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
Conversion
PnL
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
[email protected] | Data Natives 2016
Conversion
Example smart payment control
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Conversion
[email protected] | Data Natives 2016
Hint 1: create a thorough monitoring of the data inputs, e.g.
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Conversion
[email protected] | Data Natives 2016
Hint 1: create a thorough monitoring of the data inputs, e.g.
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Hint 2: make payment control dependency on article search explicit
Conversion
[email protected] | Data Natives 2016
Hint 1: create a thorough monitoring of the data inputs, e.g.
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Hint 3: document the dependencies and educate product managersHint 2: make payment control dependency on article search explicit
Conversion
[email protected] | Data Natives 2016
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
[email protected] | Data Natives 2016
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
Step 0: connect with product manager of payment control first
[email protected] | Data Natives 2016
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score
payment control
Step 0: connect with product manager of payment control first
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score
payment control
Step 2: deploy article search and collect training data for new payment control logic
Step 0: connect with product manager of payment control first
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
Step 2: deploy article search and collect training data for new payment control logicStep 3: train and deploy an updated payment control logic
Step 0: connect with product manager of payment control first
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
Step 2: deploy article search and collect training data for new payment control logicStep 3: train and deploy an updated payment control logic
Step 0: connect with product manager of payment control first
THIS MAKES PRODUCT DEVELOPMENT VERY EXPENSIVE
[email protected] | Data Natives 2016
… Excecutives:• do not underestimate the costs of data science - there is no free lunch
•prioritize features according to , where total costs cover development, testing, data acqusition, data science
•bring together product management and data science as close as possible•centralise all A/B testing & personalisation efforts in a data department
… Product manager• map out all and document all data dependencies
… Data scientist• build a thorough monitoring & alerting systems for all inputs to your logics• prefer explicit over implicit data dependencies• create sparse solutions, i.e. limit the dependencies don’t take everyone’s data• educate product management
RECOMMENDATIONS TO …
7
Thank you