Embed Size (px)
Citation preview
1 #AJLPlayNice
Playing Nice in the Product Playground
data scientists, engineers, and product managers working together to create innovative data products
Jonathan Goldman @jrgoldman
Anu Tewary @anutewary
Lucian Lita @datariver
Intuit Data Engineering & Analytics
2 #AJLPlayNice
Pop quiz …
3 #AJLPlayNice
product vision business impact success measures
effective architecture scalability & robustness metrics & monitoring
not sure what this product does, but look at the 2% lift I can get from this model... ooh, ooh, a Dirichlet prior is what this needs!! is this good for an ICML or KDD paper?
[ 1 ]
4 #AJLPlayNice
product vision business impact success measures
effective architecture scalability & robustness metrics & monitoring
not sure what this product does, but look at the 2% lift I can get from this model... ooh, ooh, a Dirichlet prior is what this needs!! is this good for an ICML or KDD paper?
Data scientists navel gazing in a corner?!
[ 1 ]
5 #AJLPlayNice
product vision business impact success measures
rapid experimentation simple models first right metrics
let’s write a new streaming framework for the weekly dashboard! we’re not meeting our SLAs, let’s write a faster json parser! let’s write an optimized distributed graph database for our data scientist.
[ 2 ]
6 #AJLPlayNice
product vision business impact success measures
rapid experimentation simple models first right metrics
let’s write a new streaming framework for the weekly dashboard! let’s write a faster json parser in Clojure! silver bullet: graph database, fp, lambda arch
[ 2 ]
Engineers reinventing the tech wheel?!
7 #AJLPlayNice
rapid experimentation simple models first right metrics
forget A/B testing, my gut tells me this is the way to go... revenue impact? Who cares! Build it anyway! no time to instrument! Let’s go to market and we’ll do that later - I’m sure that the numbers will look good!
[ 3 ]
effective architecture scalability & robustness metrics & monitoring
8 #AJLPlayNice
rapid experimentation simple models first right metrics
forget A/B testing, my gut tells me this is the way to go... revenue impact? Who cares! Build it anyway! no time to instrument! Let’s go to market and we’ll do that later - I’m sure that the numbers will look good!
[ 3 ]
effective architecture scalability & robustness metrics & monitoring
Product in a bubble?!
9 #AJLPlayNice
Data Science Jonathan
data product
Product Anu
Engineering Lucian
10 #AJLPlayNice
3
Three Steps to Risa**
2 1
** Risa is to Nirvana as Spark is to Hadoop
11 #AJLPlayNice
3 2 1
Build an Awesome Team
12 #AJLPlayNice
awesome team
13 #AJLPlayNice
never settle
14 #AJLPlayNice
find the right mix
minimum
prod
ds
eng
good
good
good
target
great
good
good
prod
ds
eng great
good
good
prod
ds
enggreat
good
great
prod
ds
eng
15 #AJLPlayNice
form pods around product
personalization & reco pod
real time data capture &
stream proc. pod
business search pod
real time commerce graph pod
16 #AJLPlayNice
blur the boundaries
17 #AJLPlayNice
3 2 1
Solve a Big Problem
18 #AJLPlayNice
Solve a big problem identify big problem
19 #AJLPlayNice
keep score
20 #AJLPlayNice
change, pivot, iterate
21 #AJLPlayNice
3 2 1
Get Out of the Way
22 #AJLPlayNice
time
trust the team to become experts
23 #AJLPlayNice
anyone can represent the team
24 #AJLPlayNice
your role as a coach?
25 #AJLPlayNice
engage!
26 #AJLPlayNice
Three Steps to Risa**
3
2
1 awesome team (pods)
solve a big problem (pods)
get out of the way (pods)
** Risa is to Nirvana as Spark is to Hadoop
27 #AJLPlayNice
Most companies are not there yet
28 #AJLPlayNice
Example 1: Multinational banking and financial services company
Took a “technology first” approach: wanted to build a hadoop cluster, because they had heard they should No product vision, but tremendous (!) possibilities Not connected closely with business needs No data science
build an awesome team solve a big problem engage
prod
ds
enggood
tiny tiny
29 #AJLPlayNice
Example 2: Large media company
Excellent engineering team Good product team, but not data driven Good metrics and beginning data science. Did not iterate quickly; data and product were too decoupled
build an awesome team solve a big problem engage
?
prod
ds
engamazing
tiny good
30 #AJLPlayNice
Example 3: Large advertising firm
Data-driven product team, but limited vision Engineering team not product focused. Could not iterate quickly Non-existent data science
build an awesome team solve a big problem engage
good
tiny
ok
prodds
eng
31 #AJLPlayNice
Example 4: Attempt at Introspection
An awesome team with data, product and engineering working together Solving hard problems – for individuals and small businesses Good metrics in place, but not there yet – more work to be 100% eyes on, hands off.
build an awesome team solve a big problem engage
32 #AJLPlayNice
3
2
1 awesome team (pods)
solve a big problem (pods)
get out of the way (pods)
33 #AJLPlayNice
Jonathan Goldman @jrgoldman
Anu Tewary @anutewary
Lucian Lita @datariver
34 #AJLPlayNice
Jonathan Goldman @jrgoldman
Anu Tewary @anutewary
Lucian Lita @datariver
35 #AJLPlayNice
Jonathan Goldman @jrgoldman
Anu Tewary @anutewary
Lucian Lita @datariver
