Leading the Product 2017 - Wendy Glasgow

Leading The Product 2017Speaker Slides

Melbourne and Sydney, Australia

Wendy GlasgowGoogle

For more information go towww.leadingtheproduct.com

Confidential + ProprietaryConfidential + Proprietary

Data: Considering more than the [email protected]

Confidential + Proprietary

Data to define what we

build

Data generated from our product

Data powering

our product

2 31



Data to define what

we build

1


Metrics to define success - moving away from the gutGet them right - we can truly build a great product that grows our business

Get them wrong - we can look successful on paper but completely miss the mark

Google Confidential and Proprietary

weight target quantity target

Russian nail factory workersEarly 20th century

Macro: What is my business trying to do? What is my team trying to do?

Build a strong profitable business

Proprietary + Confidential

Executives that adhere to metrics that tie directly tobusiness objectives

3x more likely to hit their goals

Source: New Study Reveals Why Integrated Marketing Analytics are Critical to Success, Think with Google, Forrester, March 2016


Understand the macro before diving into the micro

Product Strategy

Market Penetration | Brand Positioning | Profit Targets

Product Plan

Metrics

Marketing Str… Sales

Cost / Service External DriversEngagement

Product Teams

Your Product strategy

enables focus

Confidential + ProprietaryConfidential and Proprietary

Jack Welch, Former CEO of GE

“There are only two sources of competitive

advantage:

The ability to learn more about our customers

faster than the competition,

and the ability to turn that learning into action faster

than the competition.”


Customers are NOT created equal

Focus: Who is your customer?


Know your ‘best’ customers

$ $$$$$$$$$$

$

Value Spend less Cost to Acquire

Ideal Customer

Base

Cost toService, Support, Retain


Widen the scope of considered data

Marketing

Ad Logs Search

DM EDMSMS

CompetitionsNewsletters

Product, Websites

Stores

Engagement Analytics

Transactions

Customer Services &

Support

Call centreCustomer

interactions

Finance

TransactionBusiness

Costs

Operations & Logistics

Operational costs

Delivery

Sales

CRMPOS

Customer Value

People & Culture

People costs Skills matrix

Attrition

Tech

POSLogs

Analytics

Relevant Data

Data Strategy

Data Governance


Data generated from the product

2


● Deal with data early

● Ensure you have a data strategy

● Add a section at the definition stage

● Make it mandatory

● Provide a process, make it consistent

● Over capture and

LABEL!

You’re focused on getting your product live


We’re capturing data - what is important to consider?

● Data storage is CHEAP - $0.02 per TB per month in BigQuery!

● Capture everything possible - but make it readable

● Consider the data points you’re capturing

● Make the data meaningful - LABEL

● Link your data - IDs, Labels

● Timestamps are critical


Tools

BigQueryAttribution

Data StudioData Vis

Tag ManagerTag Mgt

OptimizeTesting/Personalisation

Google AnalyticsAnalytics



Make friends with statistics

- OR -with someone who already iscorrelation

causation

Get intimate with your data

relevance


Data as an Asset


There is no competitive advantage within an organisation!

Share your Data

There is no competitive advantage within an organisation!


Data powering our

product

3


Machine Learning is the new ground for gaining competitive edge & creating business value

*Source: MIT Survey 2017; n=375Bain Consulting Study

Competitive advantage ranked as top goal of machine-learning projects for 46% of IT

leaders & 50% of adopters can quantify ROI

2X more data-driven decisions

5X faster decisions

than others3X faster execution


Machine Learning Allows You to Solve a Problem Without Codifying the Solution

✓ Recognizes patterns in data✓ Predictive analytics at scale✓ Builds ML models seamlessly✓ Fully managed service✓Deep Learning capabilities

Google Cloud AI


First Step in This Journey Begins with Data

“Every Company will be a Data Company”

*Source: Wired, Bloomberg, Fortune, McKinsey



Machine Learning Lifecycle at a Glance

How do I collect, store and make data available to the right systems?

How do I understand what data is required to solve my business problem?

User

Data Objective

TrainServe

How do I get to a working model within the period of time where my objective is still relevant?

How do I scale prediction into production systems?

How do I keep my model relevant with continuously updated data?


Flow to build a custom ML model

Identify business problem

Develop hypothesis

Acquire + explore data

Build amodel

Train the model

Apply andscale

1 2 3 4 5 6


Structured Data● Spreadsheets, Logs, Databases● Text that includes structure● Data needs to be separated● Typical data generated from products

Unstructured Data● Natural Language, Images ● More complex but sometimes these are

better understood● Number of existing ML APIs -

Supervised Learning● Need labels on the data● Build examples to train the system

Unsupervised Learning● Data is grouped / clustered ● Drawing inferences from data sets


A feature in ML is very different from a feature in Product

In ML, a feature is an individual measurable property or characteristic of a phenomenon being observed.

Choosing informative, discriminating and independent features is a crucial step for effective algorithms in pattern recognition, classification and regression.

Feature Engineering


A feature is a data point, so what is good?

Represent raw data in a form conducive for ML

1. Should be related to the objective

2. Should be known at production-time

3. Has to be numeric with meaningful magnitude

4. Has enough examples (absolute minimum of 5)


What can I do today to plan for ML1. Find your Data Strategy and Governance owners –

get familiar with it or create it!

2. Identify the decisions your product makes today.

3. Consider suitability for automation with ML.

4. What data do you have today and what do you need to capture?

5. Capture data in line with your strategy and governance guidelines – update them if necessary.

6. Capture LOTS of data, but LABEL it well and consistently!


Takeaway 1 Takeaway 2

Value is in use of data

Think inside, outside

& future

It’s what we do with the data that mattersBUT… early consideration can increase value

How does you relate to your surroundingsRelevance, correlation and causation


● Predictive maintenance or condition monitoring

● Warranty reserve estimation● Propensity to buy● Demand forecasting● Process optimization● Telematics

Manufacturing

● Predictive inventory planning● Recommendation engines● Upsell and cross-channel

marketing● Market segmentation and

targeting● Customer ROI and lifetime value

Retail

● Alerts and diagnostics from real-time patient data

● Disease identification and risk satisfaction

● Patient triage optimization● Proactive health management● Healthcare provider sentiment

analysis

Healthcare and Life Sciences

● Aircraft scheduling● Dynamic pricing● Social media – consumer

feedback and interaction analysis

● Customer complaint resolution● Traffic patterns and

congestion management

Travel and Hospitality

● Risk analytics and regulation● Customer Segmentation● Cross-selling and up-selling● Sales and marketing

campaign management● Credit worthiness evaluation

Financial Services

● Power usage analytics● Seismic data processing● Carbon emissions and trading● Customer-specific pricing● Smart grid management● Energy demand and supply

optimization

Energy, Feedstock and Utilities

Cloud Machine Learning Use Cases

Presentations & Public Speaking

Leading the Product 2017 - Wendy Glasgow