Data mining experiences in predicting TLC churn - · PDF file · 2013-10-04Data mining experiences in predicting TLC churn Marco Richeldi ... technology to analyze the customer database

Data mining experiences in predicting TLC churn

Marco RicheldiData Mining Group

TELECOM ITALIA [email protected]

2SEUGI 20 - M. Richeldi “DM experiences in predicting TLC churn”

Agenda

• Churn management in Telcos• Developing a Churn Analysis system for fixed network

services– Business understanding– Data understanding and preparation– Modeling– Evaluation

• Conclusions


Business Scenario: Customer Orientation is key for Telcos• Most Telcos’ products and services have become commodities and

are no longer relevant for competitive advantage • Telcos are evolving a process-oriented organization via customer

relationship management (CRM) and supply chain management (SCM)

• Telcos are crafting CRM application architectures to integrate operational, analytical, and collaborative front/back-office applications

• Through 2003 enterprise marketing automation applications and call centers will converge into unified customer interaction frameworks

• The market for Analytical CRM solutions is growing rapidly across Western Europe at a compound annual growth rate of almost 50% from $0.5 billion in 1999 to $3.5 billion in 2004

• Growth of Telcos’ investment in Analytical CRM will be more moderate due to investments in UMTS technology but remains considerable


Churn management: a bottom line issue• Attracting thousands of new subscribers is worthless if an equal

number are leaving• Minimizing customer churn provides a number of benefits, such as:

– Minor investment in acquiring a new customer– Higher efficiency in network usage (having a stable base of

customers streamline network planning tasks) – Increase of added-value sales to long term customers (it is far

easier to sell additional services to an existing customer than to a new one)

– Decrease of expenditure on help desk (new customers makes more use of expensive channels to customer service)

– Decrease of exposure to frauds and bad debts (new customers are more likely to show fraudulent behavior)

– Higher confidence of investors


Churn management: scooping the problem (1)Churn can be defined and measured in different ways• “Absolute” Churn. Calculated by expressing the number of

subscribers disconnected as a percentage of the subscriber base over a given period (usually on a monthly basis)

• “Line” or “Service” Churn. Calculated by expressing the number of lines or services disconnected as a percentage of the total amount of lines or services subscribed by the customers

• “Primary Churn”. Calculated on the basis of the number of defections• “Secondary Churn”. Calculated on the basis of the drop in traffic

volume, with respect to different typology of calls


Churn management: scooping the problem (2)Measuring churn is getting more and more difficult• There is a growing tendency for Business users to split their business

between several competing fixed network operators• Similarly, carrier selection enables Residential customers to make

different kind of calls with different operators • Carrier pre-selection and Unbundling of the Local Loop may

streamline the detection of churners but makes it very difficult even for former monopolistic operators to profile customers according to their “telecommunication needs”

• Other frequent questions for Fixed Network Services: What if a customer changes his type of subscription, but remains in the same telco? What if the name of a subscriber changes? What if he relocates?


Churn rate is growing

Absolute Churn rate in Italy

~19%

~22%

Churn rate for Fixed Network Services in Italy (99/00)

International calls

Fixed To Cell phone calls

Local area calls~18%

~22%

~16%


The case study: Churn Analysis for fixed network services• The framework: a major Italian network operator willing to establish a

more effective process for implementing and measuring the performance of loyalty schemes

• Within this framework, the “churn management” project had the following goals:

– Building a new corporate Customer Data Warehouse aimed to support Marketing and Customer Care areas in their initiatives

– Developing a Churn Analysis system based upon data mining technology to analyze the customer database and predict churn


ContractsTariff plansBilling dataAccounts dataFraud / Bad debts data

Customer dataMarket dataSales dataCustomer service contacts

The churn management framework

Front-officeSystems

Marketing automation

Service automation

Salesautomation

Marketing

ListenerLoader

Loader

Loader ... ......

ETL

Data Collection &Transformation

Data Preprocessing

Data Server

Data Warehouse

Analytical Applications

Reporting OLAPData Mining

Decision Engine

Back-officeSystems

•Campaign Targets•New product / services•Loyalty schemes•Performance analysis


Business understanding (1)

• Sponsors: Marketing, IT applications, IT operations• Analysis target: Business Customers (~1M customers)• Business customers were segmented into two groups according to the

size of their account (total revenue derived)– Business: SOHO + low/medium value SME (~0.9M customers)– Top Business: High value SME + low value corporate accounts

(~0.1M customers)(alternative approaches to segmentation of customers:

• Call volume generated• Account Profitability • Potential for future development of the account)

• Two data mining tasks were carried out, one for each single group. As a result, two different models were generated


Business understanding (2)

• Churn measurement: – Business: Absolute, primary churn– Top Business: Absolute, primary churn and Service, secondary

churn• Secondary churn was defined on the basis of the volume of calls

generated using a two step approach: – Calling patterns of a small sample of loyal customers (friendly customers)

were analyzed to obtain a “loyal customer” activity pattern – A pattern matching technique was applied to the universe of customers

to detect churners

• Goal: Predict churn/no churn situation of any particular customer two/three months out from the moment the prediction is made, given 4 months of historical data. The gap is required to handle delay in data acquisition from operational sources


Data understanding

CustomerData Warehouse

Input Data•Customer demographics •Contracts•Tariff plans

!local!long distance!International!fixed to mobile calls!Internet plans

•Extra service information!special plans / rates!service bundles

•Calling patterns•Billing data•Complaint information•Fraud and bad debts data•Customer service contacts•Sales force contacts•Market data

13 operational systems

•More than 1500 indicators per customer•Extraction delay: 2 months•Loading: on a monthly basis•Size: 1.5 Tb


Data Preparation (1)

Data construction. A number of new attributes were derived to augment the informative power of source data, e.g.:

• Attributes that summarize calling patterns– Average, Stdev, …– deviations from average, computed over all customers or over customers

belonging to the same segment– Attributes summarizing the trend of calling patterns originated by each

customer over different time periods (a trend analysis technique was applied)

• Attributes that highlight temporal connections between complaints and bad debts

• Attributes that summarize the customer “service profile” and relate it to calling patterns


Data Preparation (2)

Feature selection• Exploratory analysis showed strong correlation among a number of attributes

describing tariff plans and calling patterns (=> 1/3 of attributes were discarded, leaving almost 800 attributes)

• Feature selection was carried out in two steps1. Linear techniques (regressive approach, PCA), decision trees and neural

network techniques were applied to any single group of semantically related attributes (e.g., billing attributes, attributes describing customer complaints, etc.)As a result, a subset of good predictors were selected from each group of attributes (by merging outputs of different algorithms)

2. Non linear techniques were applied to the resulting set of attributes to further reduce the dimensionality of the data set

• Output: less than 100 attributes were selected

Powered by SAS Enterprise Miner™


Modeling (1)

• The model for Business Users was built on a sample of 100k customers drawn from original data

• Source data was partitioned into training (40%), validation (30%) and test set (30%). Test set was hold out for final performance evaluation

• Log transformation were used to maximize normalization of calling patterns data

• Target variable: churn/no churn situation of any particular customer two/three months out from the moment the prediction is made, given 4 months of historical data.

Feb.JanuaryDec.Nov.OctoberSept.August

Training data

Prediction time

gap

Powered by SAS Enterprise Miner™


Modeling (2)

• The gap is required to handle the 2/3 months delay in data acquisition from operational sources

• Several SAS EM mining algorithms were applied: – Logistic regression– Decision trees

• Chi-square test• Gini reduction• boosting

– Neural networks • Back-propagation (QuickProp, Rprop) • RBF (ORBF, NRBF with softmax activation function, 1 or 2

hidden layers)– Combination of DT and NN models


Evaluation (1)

• Best predictive accuracy was obtained with a 2 hidden layer NRBF(different NRBF flavors resulted in no significant differences in predictive accuracy), but performance of DT with boosting very close

• Due to better readability, DT models were selected as the ones to be deployed

• Predictive accuracy – quite good given the hard prediction task– Top Business Users: 76% true churners, 74% true non churners– Business Users: 84% true churners, 82% true non churners

• Good gains/lift curves (lift of 3.2 and 2.6 at 10% of data for Business Users and Top Business Users, respectively)

• Best predictors: calling patterns and billing data (good performance of derived attributes summarizing trend), customer industry (for Top Business), tariff planes and extra-plan rates, complaints data (for Business users)


Evaluation (2)

• Validation tests were conducted on different data set of historical data to check the predictive robustness of resulting models

– Business user model turns out to be quite robust: its predictiveperformance drops to 70% after three months (i.e., the life span of the model is three months, after that a new one has to be generated on the new data available)

– Top Business user model is less robust: after two months its performance is not greater than 60%

• Behavior of Top Business customers changes fast and the 2-months prediction time gap severely affects performance

• This problem was in part tackled by:– Adding past model predictions as new input variables– Segmenting Tob Business customers according to their value (call

volume generated)

• As a result, decrease in performance bottoms out to 65% after two months


Conclusions

• Quite good performance except for specific groups of customers characterized by a very dynamic behavior

• Impact of prediction time gap on model performance may be hardlyreduced

• Churn analysis may be deployed quickly, provided business and data mining goals are thoroughly investigated and agreed upon. Elapsed time for this project: 2 months for each model, three persons team (1 senior data miner, 1 data miner, 1 db expert). 15% of effort forbusiness understanding, 45% for data preparation, 40% for modeling and evaluation

• Resulting models have been used to support the planning stage of a number of marketing campaigns


Telecom Italia Lab

The Mission: Monitor and manage innovation, identifying and carrying outbusiness opportunities that improve competitive position andcreate value for the Telecom Italia Group

Key activitiesR&D and Technology Integration Business FrameworksVenture CapitalIncubatorsE-learning initiatives

R&D labs (former CSELT): Turin, ItalyRole• Look-ahead role for Technology and Solutions• Excellence in key areas to support businessin the Information Society (MPEG, ADSL, UMTS)• Steer and deliver Innovation to TelecomItalia Group in Italy and abroad

ThInkLAB: TILAB’s Data Minining and Analytical CRM serviceshttp://thinklab.telecomitalialab.com/

Telecom Italia LabVia G. Reiss Romoli 274

10149 Torino – Italyhttp://www.telecomitalialab.com

Documents

Data mining experiences in predicting TLC churn - · PDF file · 2013-10-04Data mining experiences in predicting TLC churn Marco Richeldi ... technology to analyze the customer database