Faster than a Mouse: Turn Data Mining Strategy into Action Miguel Barrera, Director of Risk Analytics, Fiserv Inc. Julia Minkowski, Risk Manager, Fiserv

Faster than a Mouse: Turn Data Mining Strategy into Action

Miguel Barrera, Director of Risk Analytics, Fiserv Inc.Julia Minkowski, Risk Manager, Fiserv Inc.

© 2014 Fiserv, Inc. or its affiliates.2

Use Case : Fraud Prevention in Online Payments

Best Practices : Turning Data Mining Strategy into Action

Agenda

Use Case : Fraud Prevention in Online Payments

3

© 2014 Fiserv, Inc. or its affiliates.

Risk Analytics @ Fiserv Electronic Payments

• We prevent $200M in losses every year using data to monitor, understand and anticipate fraud

• We manage risk for $24BB in transfers, servicing 2,000+ US financial institutions, including the 5 of the top 10 banks

• A department of 5 people, we operated in start-up mode until we were acquired in 2011 by Fiserv

• We build our risk models, supervise their installation & develop the next-generation of strategies for risk mitigation


What is special about Fraud Prevention?• Fraud is performed by organized criminal groups

using sophisticated technologies and logistics

• Hard to detect: target has low frequency (2 in 10,000)

• The cost of mistakes is very high $ Losses if you fail to detect fraud 60% increase in customer attrition if you miss-classify

• The environment changes fast, so you need to adapt quickly

Fraud prevention is a great field for the application of predictive analytics


Analytics for Fraud Prevention

Explore &Understand

Anticipate& Control

Monitor


Risk Management: Goals and Constraints

Constraints:

•Build a flexible system that adapts to new fraud patterns

•Service the existing client base

•Minimize the time that the production systems will be off-line or reset

•Build the next-generation of strategies with very limited resources

Goals:

•Help the business to expand to more profitable markets (on-line booking, real time payments), while keeping loss rates constant, and customers happy


Data Miner Survey 2013 by Rexer Analytics

While 6 out 10 data miners report the data is available for analysis within days of capture, the time to deploy the models takes substantially longer. For 60% of the respondents the deployment time will range between 3 weeks and 1year.


The problem we faced…

• We had the Algorithms (SAS + Angoss)• Decision Trees + Gradient Boosting • GLM and Logistic Regression• SVM• Bayesian Estimates

• But implementation took too long…

• 3 months turn-around to estimate + deploy Logistic Regression (2008)• 1 month to estimate and deploy Trees and GLM (2010)• 1 week to estimate, 1 week to install rules in online application (2012)• 1 day to estimate and deploy Trees + GLM models (2014)


In Fraud-Mitigation Speed is the Key

How long can you wait to deploy a solution?


Why we liked Oracle Advance Analytics?

Accuracy

Agility

Scalability

• The algorithms fit are as good as more complex algorithms• The loss reduction from timely deployment (hours) compensates for

model fit

• No data transfer needed (in-database)• New opportunities to combine structured data with unstructured

data

• The integration with our DB replication makes re-fit inexpensive• The same algorithm can scale-up for all other clients


Oracle Advanced Analytics Time Value

© 2012, Oracle Corporation


Accuracy + Agility vs. Cost to Deploy

• Pick the best combination of:• Less days to deployment • High model accuracy• Lower Cost

Application Deploy (Days) Accuracy Total CostSAS Server 3 0.92 x5ODM 1 0.90 1SAS Base 15 0.83 30%Angoss 12 0.85 10%


ODM – Oracle Data Miner GUI

• Built-In in Oracle SQL Developer Tool

• Downloadable free on OTN version 4.0 or latest

• Easy to use• GUI; explore data; work-flows

• Powerful • multiple algorithms and data

transformations; 100% in-DB; build, evaluate and apply data mining models

• Deployable• Shared analytical workflows; Generates

PMML and SQL scripts for automation



ODM – Oracle Data Miner GUIOracle Data Miner Nodes




Oracle Data Miner Algorithms• Identify most important risk factors (Attribute

Importance)

• Predict Fraudsters’ behavior (Classification)

• Find profiles of bad transfers (Decision Trees)

• Predict Fraud Risk Probability (Regression)

• Segment overall population (Clustering)

• Find fraudulent transactions (Anomaly detection)

• Determine co-occuring items in baskets

(Associations)

• Reduce a large dataset into representative new attributes (Feature Extraction)


Turning Data Mining Strategy into Action


Select Best Option(s)

Success Factors and Constraints

Best Practices in Analytics

• ROI /Cost • Profitability• Operations

1. Identify Benefits & Constraints

Install into Production

•Run A/B testing

•Start Small and Increase Gradually

Data Scientist

3.Turn Strategy into Action

IT Manager

Select the Appropriate Infrastructure

•DB Architecture•Modeling techniques

2. Develop the Strategy

Provide Actionable Insights

Estimate Impact for the Business Track Benefits and KPI

• Test Predictive Models• Simulate scenarios (Monte Carlo)

Score models on KPI

Collect & Process Data•Run Descriptive Analytics•Identify patterns

BusinessManager

• Align your Team’s Incentives

Involve Key Stakeholders


Involve the Right Stakeholders

Business Manager

Data ScientistIT Manager• Preserve Service Level

Agreement• Reduce Operational Risk• Preserve Budget


Conflict of Interests?

Cannot agree on success factors?Wonder why…?


IT Manager’s Strategy

• Preserve Service Level Agreements

• Stable systems• Ease of roll-back

• Minimize Operational risk

• Control Costs


Data Scientist’s Mind

• Estimate the Best Model Possible• Improve Detection Rates• Better Algorithms, Faster Hardware• Big(ger) Data!

• Explore New Algorithms

• Put some power behind it !!

Source: Ayasdi 2014


Business Manager’s Mind

Maximize Productivity: Build for specific needs

• What is the cost? • Why does it take so long? • What is the impact on customer experience?

And: Don’t talk to me in Tech-Speak ! “First we ran a chi- square test, and then we converted the

categorical data to ordinal, next we ran a logistic regression, and then we lagged the economic data by a year…”

Source: Davenport, Tom (2013), “Keep up with your quants", Harvard Business Review, Issue July-August 2013


Communication issue


Managing the Quants

• Define clearly the objective and constraints

• Implement SMART* goal setting

•Get familiar with basic analytics concepts

•Establish a time-line for delivery then multiply x 2

•Make sure you understand enough to explain to other executives… you will champion this initiative and negotiate the budgets



Taking Care of Business(tips for Data Scientists)

• Communicate clearly business level information• When and what is the expected result• Present the key concept in 2 phrases • Avoid technical language for communication• If asked for more details, then present the “How”

• Provide a Business Dashboard• Provide the $$ metrics profit/loss reduction • Show the impact of algorithms deployed / provided • Current vs. Historical

• Pick the right model - the model that maximizes the ROI



From Paper to Execution…

Bring that Escher guy here… NOW!!


Success Factors in Fraud Mitigation

• Accuracy:• Low False Negative Rate (How much fraud $ you miss)• Low False Positive Rates (How many people you bother with

additional identity verification)

• Agility • Minimize reaction-time to fraud attacks• Make your updates easy to implement

• Scalability: • Automate processes to keep variable cost down • Invest in infrastructure and replicate across clients


Selecting the Right Tools

Easy to Use and Deploy

Can combine structured data with unstructured data - new trend

Tools that integrate in the DB • Allow for Fast Model Fitting and Re-estimation• Minimize data transport across systems

In-House Algorithms• Develop algorithms that can run directly in DB for fast estimation and

execution


Tracking Performance: Dashboard

Our dashboards tracked the key performance metrics: •Historical Trends for Fraud Rates and Losses (Business KPI)

•Percentage of Transfers affected by Risk Mitigation (Business KPI)

•% of population affected by policy and % of fraud prevented (KPI for Analytics)

•Fraud detection rates for rules installed (KPI for Analytics)


Key Takeaways

On Fraud Modeling

•When dealing with fraud, the speed to implement a new model is the most important factor

•Improvements in accuracy may be lost due to delays in deployment; systems with fast turnaround have better ROI than complex algorithms with long implementation times. Select the right tools that will enable fast analysis and deployment.

Turning Strategy into Action

•Involving the key stakeholders early in the process maximizes your chance for success. Once you have aligned the incentives for the team, selecting the appropriate techniques, tools and infrastructure becomes much simpler

•It is crucial for business managers to correctly define the problems and objectives, asking the right questions and learning the basic analytical concepts

•For data scientists it is important to select their models and projects based on the expected business impact and to translate their findings into the relevant metrics

© 2014 Fiserv, Inc. or its affiliates.38

If you have further questions or comments, please contact:

Julia Minkowski

Lead Risk Analyst, Fiserv Inc

[email protected]

408-838-3827

Source: Kdnuggets, 2012

Documents

Faster than a Mouse: Turn Data Mining Strategy into Action Miguel Barrera, Director of Risk Analytics, Fiserv Inc. Julia Minkowski, Risk Manager, Fiserv