Practical Predictive Analytics The Stepping Stones to Success

March 6, 2013

Dan Putler (Alteryx)

Cornelius Kaestner (The Boston Consulting Group)

We have shaped business thinking for 50 years...

Growth- Share Matrix Experience Curve Time-based Competition

Trading up/Trading down Change Monster Adaptive Advantage

?

Technical

Emotional

Functional

High1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005

Low

Positional durability

Manageable Data overload

Information flow

DVD,

RFID,

Digital TV,

MP3 players,

Digital cameras,

Camera phones, VoIP,

Medical imaging, Laptops,

Datacenter applications, Games,

Satellite images, GPS, ATMs, Scanners,

Sensors, Digital radio, DLP theaters, Telematics,

Peer-to-peer, Email, Instant messaging, Videoconferencing,

CAD/CAM, Toys, Industrial machines, Security systems, Appliances

DVD,

RFID,

Digital TV,

MP3 players,

Digital cameras,

Camera phones, VoIP,

Medical imaging, Laptops,

Datacenter applications, Games,

Satellite images, GPS, ATMs, Scanners,

Sensors, Digital radio, DLP theaters, Telematics,

Peer-to-peer, Email, Instant messaging, Videoconferencing,

CAD/CAM, Toys, Industrial machines, Security systems, Appliances

0

200

400

600

800

1000

1200

1400

1600

1800

2005 2006 2007 2008 2009 2010 2011

Digital Information

Moore’s Law (indexed)1

Available Storage

ExabytesClear Blurred

Industry boundaries

...including thinking on the value of Big Data

Our thought leadership resonates with our clients

2000 1998 1996 1994 1992 1990

+15%

Global revenue (Indexed, 1990=100)

2,000

1,500

1,000

500

0

2012 2010 2008 2006 2004 2002

Enterprise

Information

Management

Strategic

Analytics

Platform

Analytics

Data Business

Creation

Business Model

Transformation

Big Data

Strategy Navigation

Big Data

transformation

Advanced

analytics

Capturing value from Big Data: our framework

Alteryx supports our Strategic Analytics efforts

Key principles for our Strategic Analytics efforts

Follow demand: known challenges our clients are looking to solve

Focus on challenges with significant upside potential

Start where decent data exists

Work with clients who are open to new analytical methods

Invest where we can learn the most

Seek pragmatic, implementable solutions instead of perfectly pure analytics

Retail Example: Optimizing circulars

Approach: SKU / event promo analysis Insight and impact

Up to 50% of promotions have no impact

on sales or margin

Significant opportunities to improve value

creation, e.g.,

• 4% sales opportunity

• 7% margin opportunity

• 10% flyer cost reduction

Additional insight from the analysis

• At one retailer, stores were not consistently

executing promotions

• At another, we could identified vendors

who consistently underfunded promotions

Inc

rem

en

tal M

arg

in

Incremental Sales

Our approach for Strategic Analytics at BCG

Enable our

organization

Critical tools available to all (Alteryx, Tableau)

• 125 Alteryx users enabled in the last 6 months

Remote processing available for larger data sets

Encourage

experimentation

Seek out opportunities to test new methods

Invest in learning opportunities

Involve clients in the experimentation

Codify and

share wins

Formalize our lessons learned into products

Look for opportunities to apply products at other clients

Identify talent to

drive Strategic

Analytics

Small team to drive the effort, each with combination of skills

• Business understanding to recognize actionable solutions

• Analytical aptitude and technological savvy to leverage tools

The Lay of the Land • Predictive analytics is GREAT!...

• …but predictive analytics is a scary thought for a lot of managers

• Lots of math

• Potentially a lot of expense

• Can you believe the numbers that come out of the fancy models?

• How do you even get started?

Two Approaches to Getting Started • Hire an outside firm

• No fixed costs

• Take advantage of the outside firm’s expertise

• Do-it-yourself

• Lower variable costs

• Greater opportunities to learn and understand the capabilities and limitations of

predictive analytics

• Allows for a closer connection and integration with existing business processes

• Many organizations conduct a mixture of in-house and outsourced predictive analytics

projects

Four Steps to In-House Predictive Analytics Success • Start small and take a “learning by doing” approach

• Develop an initial list of possible predictive analytics projects that address frequent and

important business decision in your organization

• Select projects from the initial list that make use of well-known metrics for predicting

outcomes

• Compare the results of a new predictive analytics-based business process to the

incumbent process used to make a decision

Four Steps to In-House Predictive Analytics Success • Start small and take a “learning by doing” approach

• Develop an initial list of possible predictive analytics projects that address frequent and

important business decision in your organization

• Select projects from the initial list that make use of well-known metrics for predicting

outcomes

• Compare the results of a new predictive analytics-based business process to the

incumbent process used to make a decision

The Virtues of Starting Small • An initial low financial commitment with respect to both software and personnel

• You likely already have Alteryx licenses if you are in this room

• The organization is able to develop internal expertise in predictive analytics that it can

leverage in the future

• The organization develops a better understanding of what is and is not possible with

predictive analytics

• It provides the ability to assess the possible benefits from using predictive analytics to

drive business processes, but in a limited way that limits the downside risk

• Several successful small projects builds managerial confidence in the approach,

enhancing organizational buy-in

What do you Need to Start Small? • One or two current staff members with the willingness to take on a new challenge, have

a basic set of computer skills, and are given some time to experiment with the methods

• Appropriate software

• You likely already have Alteryx licenses if you are in this room

• Our Predictive Analytics – Essentials online course can provide a jump-start

• The analysis tool pack in Excel has been used by a number of organizations to get

started

• What about advanced statistical and data mining training?

• It helps, but an understanding of the business and the willingness to learn matters

more

• Asking the right question is a lot more valuable than using the best analysis method

Develop a List of Business Questions PA can Inform • A useful way to start is with your organization’s key performance indicators (KPIs) and

then determine how predictive analytics can help address the business decision that

underlie the KPIs

• OK let’s use an example to make this concrete

• Congratulations you are now the General Manager of a major league baseball team

• In this job, what are your KPIs?

• What decisions can you make in order to deliver on those KPIs?

• What information can we use to inform these decisions?

Use Well-Known Metrics to Select Projects • In many cases there are (fairly) well-known metrics that can be taken advantage of to

select projects from the list of potential projects

• Relying on others past experience in selecting predictor variables can really shorten

the time it takes to develop a useful predictive analytics model

• Recency, frequency, and monetary value (or RFM) is a well known example from direct

marketing that works well for cross selling applications to existing customers

• Web searches to find relevant articles, blog posts, slide decks, and other resources can

really help

• Should web searches fail, thinking through the information that is available at the time a

decision is made (as opposed to what is available with 20-20 hindsight) is a useful thought

experiment that can be used to develop possible metrics

Back to Baseball • We know that scoring runs is a critical element in winning baseball games, and we know

we can draft or acquire players based on statistics (metrics) that, as a team, will lead to

scored runs. What are the available statistics?

• Common baseball batting statistics on individual players available on a historical basis:

• Hits

• Walks (Base on balls and hit by a pitch)

• Strikeouts

• Singles

• Doubles

• Triples

• Homeruns

• RBIs

• Batting average / plate appearances (at bats)

Back to Baseball • The statistics are for individuals, but what happens when we combine them into a team?

• What is better, having a set of players in the batting order that can only get to first

base on a walk or hitting a single, but do so at every at bat, or a set of players in the

batting order that only hit home runs, but have a one in three chance of doing so at

each at bat?

• We just saw the common statistics, are there metrics we can construct from them that

can be more informative?

• The wisdom of Bill James and SABRmetrics: On Base Percentage + Slugging Percentage

• The question: Can we use this information as the basis for drafting or acquiring players?

Compare Models to Traditional Business Processes • Testing and experimentation are an essential part of the use of predictive analytics tools

• The goal of these tests is to objectively compare the performance of the predictive

analytics-based business process to the traditional business process

• Why?

• Many managers don’t trust models, but they are very comfortable with comparing one

group with another to see if there is a noticeable difference between the two of them

• Favorable results in these tests increase managers’ trust in predictive analytics

• How?

• A/B testing: Explicitly creating treatment (those who are addressed using a predictive

analytics-based process) and control (those who are addressed using traditional

business processes) groups and then compare results

• Retrospective testing: Use two time periods and compare differences in outcomes

based on traditional business processes and those predicted as best by a model

Thank You!