Boire Filler Group Desired Outcomes: Data Mining 1. Explain the fundamental concepts and business uses of data mining 2. Describe the critical aspects

Boire Filler GroupBoire Filler Group

Desired Outcomes: Data Desired Outcomes: Data MiningMining1.1. Explain the fundamental concepts and business uses Explain the fundamental concepts and business uses

of data miningof data mining 2.2. Describe the critical aspects of customer data for Describe the critical aspects of customer data for

marketing analyticsmarketing analytics 3.3. Understand the role of predictive modelling in Understand the role of predictive modelling in

businessbusiness4.4. Build a predictive modelBuild a predictive model5.5. Demonstrate the ability to select appropriate Demonstrate the ability to select appropriate

techniques for solving business problemstechniques for solving business problems 6.6. Understand the importance of customer segmentationUnderstand the importance of customer segmentation


What this course will NOT What this course will NOT dodo

• Teach you all the statistics you need Teach you all the statistics you need to do data miningto do data mining

• Replace real-world experience Replace real-world experience analyzing databasesanalyzing databases

Turn you into an immediate data Turn you into an immediate data miningmining practitioner practitioner


What this course will doWhat this course will do

• Help you understand how and when Help you understand how and when to use data miningto use data mining

• Assist you in talking to data miners Assist you in talking to data miners (internally or externally)(internally or externally)

• Begin your training as a data minerBegin your training as a data miner

Intro to Data MiningIntro to Data Mining

MARK2039MARK2039

Spring 2005Spring 2005

George Brown CollegeGeorge Brown College


What is Data Mining? What is Data Mining? •The process of exploration and analysis, The process of exploration and analysis, by automatic means, of large quantities of by automatic means, of large quantities of data to discover meaningful patterns and data to discover meaningful patterns and rulesrules

•What does this mean from a business What does this mean from a business standpoint ? standpoint ?

– Capitalization of above learning to maximize Capitalization of above learning to maximize ROI for a given business process.ROI for a given business process.


What is Data Mining? What is Data Mining? Continued...Continued...• Data Mining is revolutionizing business todayData Mining is revolutionizing business today

• The old business paradigms are no longer acceptableThe old business paradigms are no longer acceptable

• Companies recognize their information as a critical Companies recognize their information as a critical asset asset

• The most successful companies in the coming The most successful companies in the coming millennium will be able to intelligently utilize this millennium will be able to intelligently utilize this information for profit-maximization decisionsinformation for profit-maximization decisions


Why the Growth in Data Why the Growth in Data Mining?Mining?• Marketers are no longer revenue-driven, but ROI drivenMarketers are no longer revenue-driven, but ROI driven

• Organizations have are becoming customer centric vs. Organizations have are becoming customer centric vs. product centricproduct centric

• Too much noise and confusion in the market placeToo much noise and confusion in the market place

• Societal changes include:Societal changes include:– Consumers are time consciousConsumers are time conscious– Emphasis on quality and valueEmphasis on quality and value– Aging populationAging population– Emphasis on “What's in it for me” ?Emphasis on “What's in it for me” ?


Why the Growth in Data Why the Growth in Data Mining?Mining?• Technological ChangesTechnological Changes

– Increased storage and processing capacity within a constantly Increased storage and processing capacity within a constantly cost-reduction environmentcost-reduction environment

– Increased use of statistical tools and software for enhancing Increased use of statistical tools and software for enhancing business decision-making business decision-making

• One-to-One Marketing is becoming the “norm”One-to-One Marketing is becoming the “norm”– Increased emphasis on developing customer loyalty programs Increased emphasis on developing customer loyalty programs – Information represents a critical requirement in developing Information represents a critical requirement in developing

customer loyalty programscustomer loyalty programs– Mining the above information intelligently is the key towards Mining the above information intelligently is the key towards

successful customer loyalty programs.successful customer loyalty programs.

• The WebThe Web– Easy and timely access to large volume of data Easy and timely access to large volume of data


Data Mining as a Data Mining as a ProfessionProfession• The most important asset for successful The most important asset for successful

data mining is people.data mining is people.• Successful hiring factors to look for are:Successful hiring factors to look for are:

Quantitative skillsQuantitative skillsBusiness and problem-solving skillsBusiness and problem-solving skillsProgramming skillsProgramming skillsKnowledge of data structure, file structure, Knowledge of data structure, file structure,

system structure and their integrationsystem structure and their integrationCommunication skills and ability to liase Communication skills and ability to liase

with marketing and systems departmentswith marketing and systems departments


Common SoftwareCommon Software

• SAS (Enterprise Miner, Base SAS)SAS (Enterprise Miner, Base SAS)

• SPSSSPSS

• IBM Intelligent MinerIBM Intelligent Miner

• Angoss Knowledge StudioAngoss Knowledge Studio


Common applicationsCommon applications

• Fraud detectionFraud detection

• Direct marketingDirect marketing

• Call analysisCall analysis

• Customer segmentation Customer segmentation

• Drug testingDrug testing

• Quality controlQuality control

• Credit scoringCredit scoring

• Click stream analysisClick stream analysis


Common Marketing Common Marketing ApplicationsApplications1) 1) Acquisition of new customers.Acquisition of new customers.2) Developing Up-Sell strategies2) Developing Up-Sell strategies3) Developing Cross-Sell strategies3) Developing Cross-Sell strategies4) Reducing customer defection4) Reducing customer defection5) Creation of target customer groups for existing 5) Creation of target customer groups for existing

customer marketing programscustomer marketing programs6) Campaign management analysis6) Campaign management analysis7) Identifying high value and high potential value 7) Identifying high value and high potential value

customerscustomers8) Product affinity and bundling analysis8) Product affinity and bundling analysis9) Retail site location analysis and product distribution 9) Retail site location analysis and product distribution

analysisanalysis

One of the primary objectives of data mining is One of the primary objectives of data mining is to align marketing investment with customer to align marketing investment with customer potential. potential.


Improving Improving Business ResultsBusiness Results

• Data Mining is about identifying opportunities to improve business results. Data Mining is about identifying opportunities to improve business results. • This may be achieved by identifying segments of customers that outperform This may be achieved by identifying segments of customers that outperform

others based on certain business objectives (an objective function)others based on certain business objectives (an objective function)• For example, the results from the predictive model below identifies customers For example, the results from the predictive model below identifies customers

more or less likely to respond to a particular DM offer.more or less likely to respond to a particular DM offer.

0.0%0.5%1.0%1.5%2.0%2.5%3.0%3.5%4.0%4.5%5.0%

Decile

1

Decile

2

Decile

3

Decile

4

Decile

5

Decile

6

Decile

7

Decile

8

Decile

9

Decile

10

0.0%0.5%1.0%1.5%2.0%2.5%3.0%3.5%4.0%4.5%5.0%

Decile

1

Decile

2

Decile

3

Decile

4

Decile

5

Decile

6

Decile

7

Decile

8

Decile

9

Decile

10


Mass marketing. Same investment Mass marketing. Same investment for all customersfor all customers

HighHigh

Marketing Marketing InvestmentInvestment$/Customer$/Customer

LowLowLowLow Customer Value / PotentialCustomer Value / Potential HighHigh


Align marketing investment with Align marketing investment with customer potentialcustomer potential

HighHigh

Marketing Marketing InvestmentInvestment$/Customer$/Customer

LowLowLowLow Customer Value / PotentialCustomer Value / Potential HighHigh


Different objectives ===> Different objectives ===> Different approachesDifferent approachesDirected data miningDirected data mining When you know what you are looking for.When you know what you are looking for.

e.g. Produce a predictive model to identify e.g. Produce a predictive model to identify customers most likely to respond.customers most likely to respond.

Undirected data mining A process of discovery.

e.g. What can the data tell us about customers?


Example: Which are these?Example: Which are these?• Predicting the likelihood of response in the next Predicting the likelihood of response in the next

campaigncampaign• Analyzing call logs to determine which are Analyzing call logs to determine which are

complaintscomplaints• Determining the data mining strategy for the Determining the data mining strategy for the

next yearnext year• Why are sales decreasing in the last 3 years Why are sales decreasing in the last 3 years • Assigning a likelihood of default score on a Assigning a likelihood of default score on a

mortgage applicantmortgage applicant• Grouping customers together Grouping customers together

into segmentsinto segments


Four Stages of Data MiningFour Stages of Data Mining

Problem Identification

Creation of the Analytical Data Environment

Application of the Data Mining Tools

Implementation and Tracking

Problem Identification

Creation of the Analytical Data Environment

Application of the Data Mining Tools

Implementation and Tracking


The Data Mining Process - The Data Mining Process - Problem Problem Identification StageIdentification Stage

1)Problem Identification1)Problem Identification

Identify overallbusiness strategy

Identification and Prioritization

of business strategy components

which can be resolved through

data mining

Provide information

regarding currentdata environment

Role of Marketer Role of Data Miner Role of Systems

Example: Improve retention results. What is the data mining impact?


Conduct preliminary datadiagnostics:-source file extractions-Data Dumps-Determination of links and keys between files-Frequency distributions on

all fields on all files

The Data Mining Process: The Data Mining Process: Creation of the Analytical Creation of the Analytical FileFileRole of Marketer Role of Data Miner Role of Systems

Understand sourcesof data that are used in data mining project

Acts as Data Consultant toData Miner:-Data Dictionary-File Layouts-Star Schema-Data Nuances /Interpretations


Role of MarketerRole of Marketer

– Have clear understanding Have clear understanding of the key information of the key information within data mining solutionwithin data mining solution

– Have clear understanding Have clear understanding of how data mining of how data mining solution performs from solution performs from business perspectivebusiness perspective

– Have clear understanding Have clear understanding of how to use data mining of how to use data mining solution in future campaignsolution in future campaign

Role of Data Miner/ AnalystRole of Data Miner/ Analyst

– Design appropriate Design appropriate reports to communicate reports to communicate final data mining solution final data mining solution and its expected and its expected performanceperformance

– Consult and advise on Consult and advise on how data mining solution how data mining solution should be used and should be used and tracked in future tracked in future campaigncampaign

The Data Mining Process : The Data Mining Process : Application of Data Mining Application of Data Mining

TechniquesTechniques


The Data Mining Process : The Data Mining Process : ImplementationImplementation

Role of Marketer

Review current results of solution vs. results of solution achieved

through development

Role of Data Miner

Apply solution to database for upcoming campaign

Validate application of learning by checking

random dump of 10 records

Produce results

Role of Systems

Assist or run program to apply data mining solution to database

for upcoming campaign


What is the impact of data What is the impact of data miningmining• First Example: Increase number of orders from First Example: Increase number of orders from

100000 to 200000. Is this caused by data mining100000 to 200000. Is this caused by data mining

• Second Example: Increase the order rate per Second Example: Increase the order rate per customer from 1% to 2% with total orders customer from 1% to 2% with total orders decreasing by 100000. Is this caused by data decreasing by 100000. Is this caused by data

miningmining • A third example to illustrate the impact of data A third example to illustrate the impact of data

mining mining # of

customers # of ordersOrder Rate

Cost Per order

Scenario 1 1000000 20000 2.00% $50.00Scenario 2 100000 15000 15.00% $6.67


Problem Problem IdentificationIdentification

• How does data mining impact the business?How does data mining impact the business?– Example 1: Direct Mail Campaign to 500000 Example 1: Direct Mail Campaign to 500000

customers. Promotion cost per piece is customers. Promotion cost per piece is $1.00 $1.00

– Assume data mining can bring 10% Assume data mining can bring 10% improvement in performance for all improvement in performance for all campaigns. What is the potential data campaigns. What is the potential data mining impact here?mining impact here?

– What other metric do we need to think of ?What other metric do we need to think of ?


Problem Problem IdentificationIdentification

• How does data mining impact the business?How does data mining impact the business?– Example 1: Direct Mail Campaign to Example 1: Direct Mail Campaign to

500,000 customers. Promotion cost per 500,000 customers. Promotion cost per piece is $1.00 piece is $1.00

Note: the calculation is an opportunity cost. It calculates the additional promotional cost to achieve 5500 responders without data mining.

Scenarios # of customers Response Rate # of responders promotion costwith data mining 500,000 1.10% 5,500 $500,000.00w/o data mining 550,000 1.00% 5,500 $550,000.00

$ opportunity $50,000.00


Problem IdentificationProblem Identification– Example 2: Outbound telemarketing campaign Example 2: Outbound telemarketing campaign

to to 300,000 customers. Promotion cost person is 300,000 customers. Promotion cost person is $6.00$6.00

–Example 3: Email campaign to 1,000,000 customersExample 3: Email campaign to 1,000,000 customerswith cost per promotion of $.10with cost per promotion of $.10

Of the three examples, which campaign would you focus your data mining activities on?

Scenarios # of customers Response Rate # of responders promotion costwith data mining 300,000 1.10% 3,300 $1,800,000.00w/o data mining 330,000 1.00% 3,300 $1,980,000.00


Scenarios # of customers Response Rate # of responders promotion costwith data mining 1,000,000 1.10% 11,000 $100,000.00w/o data mining 1,100,000 1.00% 11,000 $110,000.00



Identifying data mining Identifying data mining opportunitiesopportunitieswithin your organizationwithin your organization

• Explore the organizations key business Explore the organizations key business challenges challenges

• Determine if improved customer/prospect Determine if improved customer/prospect targeting or segmentation would improve resultstargeting or segmentation would improve results

• Review the following questions:Review the following questions:

Are the overall business results reasonable?Are the overall business results reasonable? Is the product or service in a stable business environment?Is the product or service in a stable business environment? What is the current data environment?What is the current data environment? What type of budgets are available?What type of budgets are available? What type of margins does the product or service contribute to the What type of margins does the product or service contribute to the

organization?organization? How many customers or prospects do you currently target?How many customers or prospects do you currently target? Will the results of your data mining exercise be actionable based on Will the results of your data mining exercise be actionable based on

the results you are trying to improve?the results you are trying to improve?


Example 1-Identifying Example 1-Identifying Data OpportunitiesData Opportunities

• Company A has a 10,000 customers Company A has a 10,000 customers enrolled in a service that is renewed on enrolled in a service that is renewed on an annual basis. Each year only 10% of an annual basis. Each year only 10% of all customers renew their service. Their all customers renew their service. Their renewal rates for other products and renewal rates for other products and services averages 70%.services averages 70%.

Should data mining be used to improve Should data mining be used to improve retention?retention?



• Company B has a 1,000,000 customers and Company B has a 1,000,000 customers and has been cross selling a long distance phone has been cross selling a long distance phone plan for over 2 years. Over the last 6 months plan for over 2 years. Over the last 6 months acquisition results have decline and the cost acquisition results have decline and the cost per new plan member has increased beyond per new plan member has increased beyond target levels. target levels.

Should data mining be used to improve Should data mining be used to improve results?results?

• Give me an example of a data mining solution?Give me an example of a data mining solution?



• Art vs. ScienceArt vs. Science

• Retail Company collect no information on its customers. Retail Company collect no information on its customers. Market research has indicated that the key drivers of Market research has indicated that the key drivers of purchase behaviour are high income, female immigrants.purchase behaviour are high income, female immigrants.

– No individual-level information No individual-level information – Information is available only at aggregate or postal code levelInformation is available only at aggregate or postal code level– Advantages of using advanced statistical techniques are minimized Advantages of using advanced statistical techniques are minimized

within this data environment.within this data environment.– Quicker and simpler solutions will suffice.Quicker and simpler solutions will suffice.


Example 3-Identifying DataExample 3-Identifying Data OpportunitiesOpportunitiesThe Solution:The Solution:• Using an “RFM” index approach, create postal Using an “RFM” index approach, create postal

code index based on three Statistics Canada code index based on three Statistics Canada Variables:Variables:

•Median taxfiler income of postal codeMedian taxfiler income of postal code

•% of population female within postal code% of population female within postal code

•% of population landed immigrants within postal % of population landed immigrants within postal codecode IncomeIncome % Female% Female % Landed % Landed

Immig.Immig.

Average Postal Average Postal CodeCode

$40,000$40,000 52%52% 5%5%

M5A 1J2M5A 1J2 $50,000$50,000 60%60% 10%10%

IndexIndex 1.251.25 1.151.15 22

The index for M5A 1J2 is (.33 x 1.25)+(.33 x 1.15)+(.33 x 2) = 1.45


Example 3-Identifying DataExample 3-Identifying Data OpportunitiesOpportunitiesThis index scheme can then be used to score each postal code. The 800000 postal codes in Canada are then ranked into 20 half decilesbased on descending index score.

% of File

# of Postal Codes

Minimum Index in Interval

# of prospects

0-5% 40000 5.5 800005%-10% 40000 5 60000

10%-15% 40000 4.8 90000…

95%-100% 40000 0.05 30000Total 800000 3000000

How would you use this above tool?


Example 4:Example 4:

• An SVP of a large bank has spent thousands of An SVP of a large bank has spent thousands of dollars creating a credit card response model.dollars creating a credit card response model.

• The predictive model identifies those who are The predictive model identifies those who are most likely to respond to the banks next offer.most likely to respond to the banks next offer.

• The model will allow the bank to save The model will allow the bank to save considerable money – mailing only 20% of the considerable money – mailing only 20% of the prospects, they will generate 70% of all the prospects, they will generate 70% of all the responders.responders.


Example 4 (continued)Example 4 (continued)

• ““But I need the maximum number of But I need the maximum number of responders”responders”

• Attaining even 70% of the responders will Attaining even 70% of the responders will not meet the campaign expectationsnot meet the campaign expectations

• What is the real problem hereWhat is the real problem here

Data Mining is not always necessaryData Mining is not always necessary

Documents

Boire Filler Group Desired Outcomes: Data Mining 1. Explain the fundamental concepts and business uses of data mining 2. Describe the critical aspects